-- Logs begin at Fri 2022-07-29 13:36:07 UTC, end at Fri 2022-07-29 15:01:51 UTC. -- Jul 29 14:15:43.282598 dv-cinder-799871-49-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 29 14:15:43.293175 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 29 14:15:43.295158 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 29 14:15:43 2022] *** Jul 29 14:15:43.295158 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 29 14:15:43.295158 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 29 14:15:43.295158 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: nodename: dv-cinder-799871-49-iscsi Jul 29 14:15:43.295158 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: machine: x86_64 Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: clock source: unix Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: pcre jit disabled Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: detected number of CPU cores: 4 Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: current working directory: / Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: detected binary path: /usr/bin/uwsgi-core Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: your processes number limit is 47680 Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: your memory page size is 4096 bytes Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: detected max file descriptor number: 2048 Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: lock engine: pthread robust mutexes Jul 29 14:15:43.295474 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: thunder lock: enabled Jul 29 14:15:43.295956 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 29 14:15:43.295956 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 29 14:15:43.337678 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: Python main interpreter initialized at 0x55bf4a9b59c0 Jul 29 14:15:43.337678 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: python threads support enabled Jul 29 14:15:43.337678 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: your server socket listen backlog is limited to 100 connections Jul 29 14:15:43.337678 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: your mercy for graceful operations on workers is 90 seconds Jul 29 14:15:43.338047 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: mapped 403077 bytes (393 KB) for 2 cores Jul 29 14:15:43.338126 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: *** Operational MODE: preforking *** Jul 29 14:15:43.338386 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: *** uWSGI is running in multiple interpreter mode *** Jul 29 14:15:43.338386 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: spawned uWSGI master process (pid: 80317) Jul 29 14:15:43.338549 dv-cinder-799871-49-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 29 14:15:43.339710 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: spawned uWSGI worker 1 (pid: 80330, cores: 1) Jul 29 14:15:43.340296 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: spawned uWSGI worker 2 (pid: 80331, cores: 1) Jul 29 14:15:43.340555 dv-cinder-799871-49-iscsi devstack@keystone.service[80317]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 29 14:15:48.893374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 14:15:48.893374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: from cryptography.utils import int_from_bytes Jul 29 14:15:48.894214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 14:15:48.894214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: from cryptography.utils import int_from_bytes Jul 29 14:15:48.894651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 14:15:48.894651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: from cryptography.utils import int_from_bytes Jul 29 14:15:48.894786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 29 14:15:48.894786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: from cryptography.utils import int_from_bytes Jul 29 14:15:48.944508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 14:15:48.944689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 14:15:48.944777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 29 14:15:48.944854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] command line args: [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 14:15:48.944920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 29 14:15:48.945094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] command line args: [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 29 14:15:48.945199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 14:15:48.945310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ================================================================================ {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 14:15:48.945396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 29 14:15:48.945524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ================================================================================ {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 29 14:15:48.945609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] admin_token = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.945775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] admin_token = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.945866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] config_dir = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.945971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] config_dir = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] config_source = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] debug = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] config_source = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] debug = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.946799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.947949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_config_append = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_config_append = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_dir = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_dir = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_file = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_file = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.948913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.949984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.950908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.951684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] publish_errors = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] publish_errors = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.952945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] standard_threads = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] standard_threads = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.953878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] use_journal = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] use_journal = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] use_json = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] use_json = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] use_stderr = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] use_stderr = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.954853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] use_syslog = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.955000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] use_syslog = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.955075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.955147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 29 14:15:48.955316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.955401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.955549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.955638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.955803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.955887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.956974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.957966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.958847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.959976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.960955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.961969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.962897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.963985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.964942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.965962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.966938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.967864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.968903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.969942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.970875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.971922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.972912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.973948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.974985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.975952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.976866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.977937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.978964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.external = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.979926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.external = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980798 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.password = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.980885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.password = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] auth.token = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] auth.token = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.981973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.982919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.983955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.984925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.985920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.986900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.987994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.988946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.989044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.989199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.989791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.989966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.990991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.991939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.992877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.993998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.994957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.995943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.996876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997047 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.997989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.998991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:48.999900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.000966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.001933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.002991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.003937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.004855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.005588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.005682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.005827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.006920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.007950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.008892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.009871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.010938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.011944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.012904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.013886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] role.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] role.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] role.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] role.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.014972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.015931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.016955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.017937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.018971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.020889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.021897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.022963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.023992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.024913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.025765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.026901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.connection = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.connection = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.027980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.028871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.029923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.030970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.031894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.032024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.032155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.032155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80330) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 14:15:49.032417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 29 14:15:49.032417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80331) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 29 14:15:49.060891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.061070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.061171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.061257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.061335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.061455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.070313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.070461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.100752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.100970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.101167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.101374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.158105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.158456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.209295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.209533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.209675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.209829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.274674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.274890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.275991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.276128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.276128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.299968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.300876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.301042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.312155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.312319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.325440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.325666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.325814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.325950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.326897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.327028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80330) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.327161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.327161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80331) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 29 14:15:49.343731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.343938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 14:15:49.344070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.344205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 14:15:49.344337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.344483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 14:15:49.344617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.344761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 29 14:15:49.344894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.345709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.355749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.355955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.356779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.363435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.363575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.368231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.368377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.368506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.368652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.368786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.368918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.369869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.370935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.371075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.393631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.393781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.395095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.395240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.395375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.396051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.400491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.400765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.400850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.401059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.401158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.401405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.407778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.408017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.408113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.408295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.412768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.413070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.413298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.413391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.413582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.413688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.413875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.413966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.414948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.415219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.430564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.430840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.431312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.431601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.435001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.435767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.438141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.438433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.438931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.439224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.442810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.443641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.445977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.446836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.449138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.449431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.449711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.449989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.450824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.451113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.451400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.467040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.467322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.467590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.467850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.467942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.468145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.468411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.468673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.475622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.476115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.477159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.477432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.477687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.477960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.480218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.480495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.480747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.480840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.481962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.492897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.493203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.493360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.493484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.493639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.493755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.493918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.494747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.495020 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.510246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.510634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.513220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.513619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.516157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.516551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.518608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.519041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.520174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.520455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.521168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.521308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.521437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.521745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.527705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.528261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.529318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.529875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.532210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.532495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.532772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.532853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.533072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.533174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.533358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.533628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.541590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.541873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.542156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.542294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.542580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.542857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.548907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80331) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.549206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.549478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.549736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80330) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 29 14:15:49.549821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.550900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.551789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80331) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.552916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.553214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.553481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80330) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 29 14:15:49.586344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bf4a9b59c0 pid: 80331 (default app) Jul 29 14:15:49.588274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bf4a9b59c0 pid: 80330 (default app) Jul 29 14:15:49.589611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7df0a-d2b4-4214-8333-d2dc5d15d45a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:15:49.589897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7df0a-d2b4-4214-8333-d2dc5d15d45a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:15:49.590169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7df0a-d2b4-4214-8333-d2dc5d15d45a None None] PATH_INFO: `/v3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:15:49.594441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Fri Jul 29 14:15:49 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:15:57.937220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:15:57.937813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:15:57.937813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:15:57.942203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:15:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:15:57.948100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:15:57.948514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:15:57.948735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:15:58.020616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: INFO dbcounter [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] Registered counter for database keystone Jul 29 14:15:58.042405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG oslo_db.sqlalchemy.engines [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80331) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 14:15:58.048800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writer thread running {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 14:15:58.233935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG passlib.handlers.bcrypt [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80331) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 14:15:58.234291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG passlib.handlers.bcrypt [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80331) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 14:15:58.288906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:15:58.290561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:15:58.417152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:15:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 29 14:15:58.550365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:15:58.619372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: INFO dbcounter [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] Registered counter for database keystone Jul 29 14:15:58.637894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG oslo_db.sqlalchemy.engines [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80330) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 29 14:15:58.644074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writer thread running {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 29 14:15:58.762896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ff649-45e1-405f-902e-00ee912b8564 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:15:58.767076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:15:58.767714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:15:58.767993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:15:58.768267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:15:58.776921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:15:58.828170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 14:15:58.828452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc483f1-c65d-4daa-ae48-2171be74f6db admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:15:58.838453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Jul 29 14:15:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:15:58.844332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:15:58.858557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769bde3-4cec-42cd-bb7a-203d57334021 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:15:58.862911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:15:58.864026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:15:58.864306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:15:58.864577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] PATH_INFO: `/v3/projects/70f7d9422ed44219ac78a91471e7dbb9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:15:58.866071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] RBAC: Authorizing `identity:get_project(project_id=70f7d9422ed44219ac78a91471e7dbb9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:15:58.916205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING keystone.common.rbac_enforcer.enforcer [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 29 14:15:58.916548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098dd53a-232d-4839-8973-6c324d4b03d8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:15:58.921087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Fri Jul 29 14:15:58 2022] GET /identity/v3/projects/70f7d9422ed44219ac78a91471e7dbb9 => generated 344 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:00.907339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be16ec-7a08-47f5-9083-9f3e11a7a6df None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:00.907853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be16ec-7a08-47f5-9083-9f3e11a7a6df None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:00.907935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be16ec-7a08-47f5-9083-9f3e11a7a6df None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:00.912062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:00.918208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:00.918665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:00.918759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:00.938781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:00.940296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:00.948673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 29 14:16:00.970887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be16ec-7a08-47f5-9083-9f3e11a7a6df None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:00.982850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be16ec-7a08-47f5-9083-9f3e11a7a6df None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:00.986981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:00.987584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:00.987865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:00.988141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:00.995871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:00.997674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f24b-85b8-474a-9182-4f2bcd70bb17 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:01.006312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Jul 29 14:16:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:16:01.011982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:01.020887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f40eea-3d1f-4c8d-ba03-37672aa382ba None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:01.026012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:01.026636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:01.026919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:01.027202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:01.028578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:01.030450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063fa4e-f292-4f51-9bef-0d2a83134703 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:01.031987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Fri Jul 29 14:16:01 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:03.079974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3104-5367-43ee-b75d-d0beb0e9b44a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.080519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3104-5367-43ee-b75d-d0beb0e9b44a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.080519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3104-5367-43ee-b75d-d0beb0e9b44a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.081892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:03.087260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.087690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.087940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.107026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:03.108218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:03.109054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e829c-877a-4050-8be4-2793561f01b4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.109346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e829c-877a-4050-8be4-2793561f01b4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.109621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e829c-877a-4050-8be4-2793561f01b4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.110946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:03.115895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.116329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.117209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.160229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG passlib.handlers.bcrypt [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80330) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 29 14:16:03.160659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG passlib.handlers.bcrypt [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80330) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 29 14:16:03.192014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:03.206817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:03.208397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:03.215196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.239506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.309616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:03.321443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a74b0-14b8-4dca-a7e3-27129150f273 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.331257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.355274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.355922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.356202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.356474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.367820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:03.377062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:03.378020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:03.378810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:03.386775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.432170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c3ab8-2d10-4c94-98b0-53bbf81d0220 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.434754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.468790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.469736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.470622 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.470908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.471187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.488053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d223-8308-4c00-9275-5c455ff90a09 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.522670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.523366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.523648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.523927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.524644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:03.524945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.526369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.537862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:03.538805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.540437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.543985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.638248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 14:16:03.643432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.644064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485f92-877d-4c4b-a308-218b472ea8de None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.646064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.750365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c922a0-2f5a-4863-9283-b886acb0da27 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.785726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.786035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.786368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.786649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.786878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.786964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.787151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.787424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.788088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.789587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.791164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:03.796500 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.807298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.809059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.812193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:16:03.827596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:03.828463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:03.829311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:16:03.829311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:16:03.829311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:16:03.829311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:03.829311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:03.834652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.890257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:03.900035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb4b4-fd38-433a-90d9-b871d99c77f9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.933248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb21697-4242-4eb1-8226-de7deefc610d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:03.934304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.934952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.935235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.935513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.936676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.938147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.939720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:03.946057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:03.967139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:03.967758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:03.968041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:03.968319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:03.969033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:03.969342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:03.970724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318f9ee-1eb3-46f6-b416-1b083331bac4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:03.988202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:04.029211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:04.050345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf587eb9-74b0-4cbd-8c4c-d54b7da35b2b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:04.086807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:04.087448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:04.087981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:04.087981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:04.088790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.domain.id=default, user.id=4c8c97aa5a724434b6ecf6c70f044b44) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:04.089128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:04.090607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b17df-03f8-447b-a9a7-7bbe607114f2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:04.098146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Jul 29 14:16:03 2022] GET /identity/v3/role_assignments?user.id=4c8c97aa5a724434b6ecf6c70f044b44&scope.domain.id=default&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:06.303414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6306c9-d6a5-4ce5-bd72-250450f44dfd None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.303967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6306c9-d6a5-4ce5-bd72-250450f44dfd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.304125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6306c9-d6a5-4ce5-bd72-250450f44dfd None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.305401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:06.311093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2672-60f9-4faa-81e0-7c62997785a4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.311514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2672-60f9-4faa-81e0-7c62997785a4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.311684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2672-60f9-4faa-81e0-7c62997785a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.338017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2b0f2672-60f9-4faa-81e0-7c62997785a4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:06.339235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2b0f2672-60f9-4faa-81e0-7c62997785a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:06.431603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:06.466536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6306c9-d6a5-4ce5-bd72-250450f44dfd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:06.525275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410e08-51ed-4e82-81b9-fec4cfcc06a2 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.525598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410e08-51ed-4e82-81b9-fec4cfcc06a2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.525835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410e08-51ed-4e82-81b9-fec4cfcc06a2 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.527210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:06.532574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a87fb-9087-4527-816a-572ba03cfe0f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.533165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a87fb-9087-4527-816a-572ba03cfe0f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.533651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a87fb-9087-4527-816a-572ba03cfe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.552449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ce0a87fb-9087-4527-816a-572ba03cfe0f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:06.554510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ce0a87fb-9087-4527-816a-572ba03cfe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:06.626414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 14/25] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:06.631870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f20d-7ba8-40dd-9ddc-7412203a1f23 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.632917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f20d-7ba8-40dd-9ddc-7412203a1f23 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.633508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f20d-7ba8-40dd-9ddc-7412203a1f23 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.635165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 15/26] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:06.640080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.640575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.641078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.646228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6306c9-d6a5-4ce5-bd72-250450f44dfd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:06.651415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:06.667074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:06.668505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:06.676847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:06.706580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:06.707161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.707441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.707716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.733195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:06.739968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:06.740846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:06.742195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:06.750013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 12/27] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:06.750887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:06.755122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 16/28] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:06.756036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:06.862878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e7c3d-2b48-4c97-bdad-1f9a04a9d020 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:06.901145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:06.901798 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.902091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.902369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.903123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:06.903432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:06.904890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:06.906724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:06.907119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01ca5d-c251-45a9-be8d-a35be10c5303 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:06.923300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 17/29] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:06.925259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:06.948905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:06.949906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:06.950419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:06.950910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:06.951866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:06.953509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.041897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5ad5c-14f9-4ea3-834d-70547abf6074 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.083573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.084245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.084530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.084859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.098807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.100471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:16:07.118477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:07.119378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:07.120223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:16:07.120223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:16:07.120223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:16:07.120223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:07.120223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 18/30] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:07.120943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.187010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:07.242378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79e6-da84-4965-bb71-83b1c656568b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.282623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.283504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.283960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.284384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.285754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.287363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bfd2d-a1c3-4b27-afda-284099225274 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.289318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 19/31] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:07 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:07.290635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6153-ac9a-4305-8c43-85bff75b5766 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.291069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6153-ac9a-4305-8c43-85bff75b5766 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.291503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd6153-ac9a-4305-8c43-85bff75b5766 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.292920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 20/32] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:07.293832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd6153-ac9a-4305-8c43-85bff75b5766 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.307868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 13/33] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 14:16:06 2022] POST /identity/v3/roles => generated 219 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:07.309689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.352856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:07.431685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd6153-ac9a-4305-8c43-85bff75b5766 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.455442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:07.477553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.478813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.479837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.480315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.481425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:07.481949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.483984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d076-4f1f-442b-a941-98d233c6e7bc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.488898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc026c-f91c-4bc5-8c70-0a26ec66801c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.528660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 21/34] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:07.530048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.530334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.530628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.534400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.535111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.535403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.535681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.537394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.539937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.541927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 14/35] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:07.542799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.562853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:07.564005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:07.655864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 22/36] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:07.656939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.662977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f190-605c-4c3b-b1f2-76dfcb79ebd0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.681352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:07.709223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.709945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.710801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.711098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.711982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.713489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.782250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ca372-88a4-4181-8045-52d9e0cb825c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.794091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:07.798816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 15/37] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:16:07 2022] POST /identity/v3/projects => generated 301 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:07.800315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:07.827269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.827961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.828249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.828532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] PATH_INFO: `/v3/domains/default/users/4c8c97aa5a724434b6ecf6c70f044b44/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.831273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4c8c97aa5a724434b6ecf6c70f044b44, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.832699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1943b23-edb9-4dce-9546-f1972b721e02 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:07.937293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055161d-e5d4-4006-99e3-0b692e313edc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:07.978099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:07.978712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:07.978993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:07.979269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:07.980044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:07.981511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e97030-9060-4220-b98d-d86ef7273b5e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:08.062763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 16/38] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 14:16:07 2022] POST /identity/v3/roles => generated 225 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:08.063917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a87f-a866-4e41-8e9e-3e11de7cffe4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.064416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a87f-a866-4e41-8e9e-3e11de7cffe4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.064704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a87f-a866-4e41-8e9e-3e11de7cffe4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.066624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 23/39] 10.222.0.49 () {58 vars in 1361 bytes} [Fri Jul 29 14:16:07 2022] PUT /identity/v3/domains/default/users/4c8c97aa5a724434b6ecf6c70f044b44/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:08.070635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 17/40] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:08.072118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.072246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfc84-be7f-4235-b2c7-22194b5b3899 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.079431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.079581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfc84-be7f-4235-b2c7-22194b5b3899 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.079666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.079746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfc84-be7f-4235-b2c7-22194b5b3899 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.079746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 24/41] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:08.083624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.083915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.084197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.095636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:08.097305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:08.105740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:08.107362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:08.113448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:08.197887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 18/42] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:08.219643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 25/43] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:08.226977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:08.243971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:08.356317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941066ad-905f-4254-8e8b-9935cf78edce None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:08.370633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:08.377464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a915b-145c-4d1b-af85-be2916612b15 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:08.396016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:08.396677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.396972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.397273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.399570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:08.401071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:08.402879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 19/44] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:08.408282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:08.414778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:08.415429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.415710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.415984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.416753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:08.418358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3086b9-35d1-4079-beee-89f1e6bff8d2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:08.425257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:08.515313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c647cd-fd74-498f-b33c-bcc823ca292d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:08.525346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 26/45] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 14:16:08 2022] POST /identity/v3/roles => generated 223 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:08.567828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:08.568462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:08.568742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:08.569038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:08.570748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:08.572439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443a5ae-9413-46f6-8892-da88706f1ddb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:08.650007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 20/46] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:16:08 2022] POST /identity/v3/projects => generated 312 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:16:10.129434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08fbbf-a967-4de6-a378-661d53391c27 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.129949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08fbbf-a967-4de6-a378-661d53391c27 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.130032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08fbbf-a967-4de6-a378-661d53391c27 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.131353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 27/47] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:10.137032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.137435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.137583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.156152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:10.157382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:10.216018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:10.223227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 21/48] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:10.245612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08fbbf-a967-4de6-a378-661d53391c27 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.348392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08fbbf-a967-4de6-a378-661d53391c27 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:10.383236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:10.383913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.384207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.384523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.398028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:10.403087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:10.403912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:10.404654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:10.405902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 28/49] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:10.410026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.515233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616649c-b786-4c34-8b14-19a1074571a4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:10.549603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:10.550236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.550522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.550800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.551528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:10.551838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:10.553270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898d33-d31c-4ec1-8856-981812c36ffa None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:10.566778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 22/50] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:10.573022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.591711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:10.647983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f88481-2b9f-472d-b5db-2e98968449d8 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.648273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f88481-2b9f-472d-b5db-2e98968449d8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.648680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f88481-2b9f-472d-b5db-2e98968449d8 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.650214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 23/51] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:10.657033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.657033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.657033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.674067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:10.676187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:10.680675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8f5a-d0c1-4393-b3a4-68983281fff3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:10.717919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:10.718570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.718858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.719140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.720318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:10.721979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:10.723601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 29/52] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:10.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.734100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:10.747977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 24/53] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:10.770421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.814197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:10.829429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6744ee-4f66-4a34-9236-95eb51a59e64 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:10.864067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:10.864721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.865029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.865334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.867079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:10.868134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff0aad-a583-4c22-9710-2b4cba3b12e3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:10.868558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:10.870195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 30/54] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:10.875812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.902124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:10.902703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:10.902987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:10.903267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:10.904423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:10.905945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:10.907527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 25/55] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:10.913031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:10.969370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:10.978192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e25a-637d-4aa2-a1e1-c1ab0f556b9b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:11.012335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:11.012952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:11.013266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:11.013429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a765a-afb8-445b-88a1-12f169b89eae None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:11.013543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:11.014336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.domain.id=default, user.id=4c8c97aa5a724434b6ecf6c70f044b44) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:11.014648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:11.016108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8423f9-f44f-47a0-9f0c-df2558eb21ad None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:11.021857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 31/56] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Jul 29 14:16:10 2022] GET /identity/v3/role_assignments?user.id=4c8c97aa5a724434b6ecf6c70f044b44&scope.domain.id=default&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:11.050155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:11.050808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:11.051094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:11.051373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:11.052079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:11.053670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa26898-48a8-44a0-a611-09ea0ae25ef2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:11.165775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 26/57] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Jul 29 14:16:10 2022] POST /identity/v3/projects => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:16:13.144566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812b79-57f9-47f9-8a3b-00250bc0beb7 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:13.145109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812b79-57f9-47f9-8a3b-00250bc0beb7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:13.145222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812b79-57f9-47f9-8a3b-00250bc0beb7 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:13.146498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 32/58] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:13.151705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:13.152090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:13.152259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:13.170584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:13.171747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:13.236328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 27/59] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:13.260506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7812b79-57f9-47f9-8a3b-00250bc0beb7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:13.278779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:13.363027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7812b79-57f9-47f9-8a3b-00250bc0beb7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:13.395792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:13.396356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:13.396633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:13.396905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:13.399055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:13.400506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef152af-dd81-48a3-8ea8-ad95e5ecd923 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:13.402095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 33/60] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:13.407650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:13.446021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:13.503411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe4aa-e441-486b-8161-7ffb93ec10ba None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:13.536727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:13.537375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:13.537656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:13.537930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:13.539260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:13.540709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbc327-4b6e-43fe-818b-59c24140f8d0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:13.635117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 28/61] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:13 2022] POST /identity/v3/users => generated 275 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:16.706434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd4f2-54ed-4308-ac3b-b88608e5d690 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:16.706434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd4f2-54ed-4308-ac3b-b88608e5d690 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:16.706434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd4f2-54ed-4308-ac3b-b88608e5d690 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:16.706434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 34/62] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:16.712485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fe18-9d8f-4c2c-b59d-b695be558d82 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:16.713340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fe18-9d8f-4c2c-b59d-b695be558d82 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:16.713841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490fe18-9d8f-4c2c-b59d-b695be558d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:16.741093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d490fe18-9d8f-4c2c-b59d-b695be558d82 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:16.742967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d490fe18-9d8f-4c2c-b59d-b695be558d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:16.842363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 29/63] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:16.877428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafd4f2-54ed-4308-ac3b-b88608e5d690 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:16.951023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6254-1979-427e-b5d4-4eed69fbee02 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:16.951674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6254-1979-427e-b5d4-4eed69fbee02 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:16.952150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6254-1979-427e-b5d4-4eed69fbee02 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:16.953743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 30/64] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:16.958890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:16.959417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:16.959923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:16.994574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:16.998386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:16.999606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:17.025379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.035111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafd4f2-54ed-4308-ac3b-b88608e5d690 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.077050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 31/65] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:17.105102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.105763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.106046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.106322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.118987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-03e18b72-c1be-4a4d-b27a-15ce82cb3eac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:17.127296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:17.128162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:17.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 35/66] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:17.132252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.136204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd673e-9c7c-416d-ad80-bf5d258f832c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.136495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd673e-9c7c-416d-ad80-bf5d258f832c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.136776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd673e-9c7c-416d-ad80-bf5d258f832c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.138809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 36/67] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:17.139384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd673e-9c7c-416d-ad80-bf5d258f832c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.265939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af7ce3-6a80-4415-9ffa-4f74a65e38ea None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.282905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd673e-9c7c-416d-ad80-bf5d258f832c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.302205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.302855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.303136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.303414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.315163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1e566828-b8f8-4e5a-ba5f-e67c63cd89e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:17.321015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:17.321930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:17.322705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 32/68] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:17.324188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6553-1964-407d-a258-477941233c3f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.324479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6553-1964-407d-a258-477941233c3f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.324757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6553-1964-407d-a258-477941233c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.328484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.329152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.329442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.329725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.330490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:17.330802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.332201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b4e44-7c79-41a0-8f47-367c74956d59 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:17.341322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 37/69] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:17.342726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-66ee6553-1964-407d-a258-477941233c3f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:17.343746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89211-95f2-4704-9b2d-6ce0bd9b4ff3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.343911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-66ee6553-1964-407d-a258-477941233c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:17.344058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89211-95f2-4704-9b2d-6ce0bd9b4ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.344323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89211-95f2-4704-9b2d-6ce0bd9b4ff3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.346023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 38/70] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:17.347908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552babc-1371-4984-b746-1e5f0cbbbc01 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.348195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552babc-1371-4984-b746-1e5f0cbbbc01 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.348471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552babc-1371-4984-b746-1e5f0cbbbc01 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.349726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 39/71] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:17.350910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552babc-1371-4984-b746-1e5f0cbbbc01 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.356347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.360693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.411848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 33/72] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:17.412770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee6553-1964-407d-a258-477941233c3f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.457008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552babc-1371-4984-b746-1e5f0cbbbc01 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.491396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.492611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.492897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.493205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.493926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:17.494238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.495623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745ca2-cbed-4cc8-a418-62e88d4d3dc6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:17.504139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 40/73] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:17.505331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.505622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.505902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.519068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee6553-1964-407d-a258-477941233c3f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.525588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:17.526798 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:17.552823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.553497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.553781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.554170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.571198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.572653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e0d65-9f93-4035-9e76-a3db8dea32ca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:17.574348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 34/74] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:17.578169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.578457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.578876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.584921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.588967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.601945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 41/75] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:17.601945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.602175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:17.602175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:17.666841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 35/76] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:17.667697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.693546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e16a08-83ab-4c5c-b4df-3ad63c47d60c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.727871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.728724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.729096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.729376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.740086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:17.745042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:17.745902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:17.746740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 42/77] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:17.747421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.770480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432a0d0-be0b-4b90-a5ab-af92c4a5a96e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.800175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.805391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.806038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.806324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.806481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:17.806584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.807921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.809672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:17.811271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 36/78] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:17.812088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.852489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d14f1-56f9-4503-ab29-eb918a04635d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.887141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.887761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.888046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.888326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.910995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5beb5-d2c3-4b41-a4ed-c34cc0ea2834 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:17.944711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:17.945334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:17.945614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:17.945889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:17.947041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.948476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:17.954491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 37/79] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:17.954491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:17.957158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:17.958666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.014510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.016511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.050239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24516b82-5daf-48bc-afbf-500de333edcb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.057035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 43/80] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:18.057859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.086390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.087040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.087326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.087606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.097554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:18.103147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:18.104075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:18.104826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 38/81] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:18.105514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.161073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd255307-5973-4c79-8c1c-dfd58b3c7b3c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.196639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.197306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.197597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.197881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.198893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:18.198893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.200286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.204280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1994c-39a7-4048-b69a-eb008cd61a8a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.208263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 44/82] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.209058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.232646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.233030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.240300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.240929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.241251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.241690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.330183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cff45-b5e8-4762-a932-281e14c39250 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.331632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.333669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.368801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.369448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.369735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.370012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.370696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.372150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.437087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 39/83] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:18.449021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 45/84] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:18 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:18.449021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.449336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.474293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.546952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.559725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce50e-3ecd-4482-b60d-f2c901ac6403 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.571669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2932167-6867-43ba-bf1e-d858f7aa6056 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.602271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.602875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.603159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.603780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.604621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.606122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.607975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 40/85] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.612947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.618515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.619164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.619453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.619740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.621324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:18.621324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.625446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.633505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 46/86] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.634376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.739590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4907482-cd91-4229-91ef-3346c0d64530 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.741213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.765385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30614e-3e8e-4759-847f-8fcfdfaac773 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.787371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.788018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.788374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.788899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.790174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.791630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.793291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 41/87] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.794058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.806922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.808124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.808675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.808968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.810619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.812807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.815536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 47/88] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.816436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.828118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:18.909170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3d35a-0845-431d-866b-c56f4851ef7b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.946072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.946898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.947178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:18.947522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:18.948351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:18.948660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:18.950200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:18.954741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d142a-daf9-4a2a-9200-737481fc3c73 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:18.959312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 42/89] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:18.960292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:18.998999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:18.999674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:18.999954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:19.000228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:19.001620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:19.003210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:19.004949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 48/90] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:19.005768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:19.019120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:19.127528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609c01-a03c-4ecf-b6b7-c0460f4af248 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:19.161759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:19.198363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:19.199823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:19.200742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:19.201299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:19.203954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec57712-0e9a-442c-94c5-79ee09e16120 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:19.265694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:19.266383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:19.266669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:19.266944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:19.267749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=4c8c97aa5a724434b6ecf6c70f044b44) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:19.268060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:19.269553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:19.282175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 49/91] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:19 2022] GET /identity/v3/role_assignments?user.id=4c8c97aa5a724434b6ecf6c70f044b44&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:19.283847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:19.328781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:19.330318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:19.386188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:19.497759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5eec1-5aaf-4ebd-a59d-2093c8f62257 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:19.540295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 43/92] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:19.546898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:19.589504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:19.590525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:19.591012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:19.591493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:19.615337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:19.684232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a3-a5d4-48df-ab24-3ca9498212df None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:19.713805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:19.731755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:19.732422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:19.732731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:19.733022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:19.734500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:19.736245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:19.743714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 44/93] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:19 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:19.745948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:19.771126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:19.773539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:19.939465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba782c34-cf50-413f-9aba-59c4ed3cc084 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:20.018277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:20.018950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:20.019234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:20.019579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:20.020389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:20.020702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:20.022234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:20.034709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 45/94] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:19 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=2d3d839c7e5143618bb567643903898d => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:20.037820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 50/95] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:20.043839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:20.159927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:20.202839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1203c1-7ea0-42e5-8be2-48bce38ad491 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:20.241460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e284bf-4603-499f-b895-23890339e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:20.242271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e284bf-4603-499f-b895-23890339e93e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:20.242716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e284bf-4603-499f-b895-23890339e93e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:20.243068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e284bf-4603-499f-b895-23890339e93e None admin] PATH_INFO: `/v3/projects/baf2743cac19469aa446f2a30e8e7b26` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:20.244310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e284bf-4603-499f-b895-23890339e93e None admin] RBAC: Authorizing `identity:get_project(project_id=baf2743cac19469aa446f2a30e8e7b26)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:20.245792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e284bf-4603-499f-b895-23890339e93e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:20.247620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 46/96] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:20 2022] GET /identity/v3/projects/baf2743cac19469aa446f2a30e8e7b26 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:20.253650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:20.326193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:20.411364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df64fd-22f6-41f7-884f-a0fcfd136eb9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:20.477936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:20.478601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:20.478891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:20.479184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:20.480075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=baf2743cac19469aa446f2a30e8e7b26, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:20.480398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:20.481938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23419b-a82a-433c-b9ca-173da0d7cfb8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:20.488382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 51/97] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:20 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=baf2743cac19469aa446f2a30e8e7b26&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:21.037613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee05f8-cf11-4378-a1c8-4502ad8913d8 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.038121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee05f8-cf11-4378-a1c8-4502ad8913d8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.038238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee05f8-cf11-4378-a1c8-4502ad8913d8 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.040040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 47/98] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:21.055160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.055653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.055809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.086577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:21.087804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:21.186719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 52/99] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:21.228110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee05f8-cf11-4378-a1c8-4502ad8913d8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:21.343732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee05f8-cf11-4378-a1c8-4502ad8913d8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:21.371964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:21.380154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:21.380811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.381129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.381412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.382685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:21.384120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cbeeb-7344-496b-aa1e-d9884277b003 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:21.385831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 48/100] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:21.393130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:21.435490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:21.456342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970dfb-d7e8-42d8-bbd1-82023253bc54 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.457024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970dfb-d7e8-42d8-bbd1-82023253bc54 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.457512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970dfb-d7e8-42d8-bbd1-82023253bc54 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.459077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 49/101] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:21.467166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4099-6077-4ce6-bd55-822634caee39 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.467719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4099-6077-4ce6-bd55-822634caee39 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.468304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4099-6077-4ce6-bd55-822634caee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.514438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0a8d4099-6077-4ce6-bd55-822634caee39 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:21.516958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0a8d4099-6077-4ce6-bd55-822634caee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:21.591376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272408-1629-4e0b-b656-dc368a8e6620 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:21.625813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:21.626929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.627407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.628158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.629119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:21.631243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e27d3-8799-4825-b327-baa08aa97508 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:21.657019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 50/102] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:21.690006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1ad7-d16e-4861-bbad-e070b33c0b90 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.690613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1ad7-d16e-4861-bbad-e070b33c0b90 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.691108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace1ad7-d16e-4861-bbad-e070b33c0b90 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.692665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 51/103] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:21.695041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace1ad7-d16e-4861-bbad-e070b33c0b90 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:21.742594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 53/104] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:16:21 2022] POST /identity/v3/projects => generated 302 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:21.744269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.744805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.745310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:21.788937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:21.790543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:21.873354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 54/105] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:21.901750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:21.910064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace1ad7-d16e-4861-bbad-e070b33c0b90 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:21.920563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:21.980384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:21.997891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:21.998640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:21.998924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:21.999206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.023663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:22.028903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:22.029776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:22.030564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 52/106] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:22.039521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.053652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63e3f9-9491-4c00-8e0c-1cfc3275f7c7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.146656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.147450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.147700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.148002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.182299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:22.190676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462abf3f-4f3b-48f6-90db-b6ff0283a3fd None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:22.196985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:22.197870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:22.198628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 55/107] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:22.209784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.240747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.242484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.242981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.243450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.244393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:22.244889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:22.246691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:22.260314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 53/108] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:22.277188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.312819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:22.356924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:22.391307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be0be-b41c-4b37-a560-8d91f670061d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.442287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2d38-f7d9-4569-9f59-2a3d149e87cb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.476817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.477481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.480935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.481253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.482015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:22.482332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:22.483763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:22.508885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.509610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.509900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.510188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] PATH_INFO: `/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.510978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 56/109] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:22.518660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.535072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] RBAC: Authorizing `identity:get_user(user_id=0edc6a0ed6ed40ff9505011e5dea4e37)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:22.536577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:22.541710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 54/110] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:22.544310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.696825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284730d-bec4-4592-a275-75ed6ed55cbf None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.712231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:22.748267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.748902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.749219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.749503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.846980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889ff8b-363d-4053-ab80-6159fdae0320 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:22.888240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:22.890009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4f05a-4d9a-4a72-974b-d41eced01679 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:22.895215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:22.895871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.896155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.896433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.898800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:22.899520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bce776-42e1-4ada-a380-6d46f95f3f83 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:22.901681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 57/111] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:22.902957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a7c6-397d-441c-825f-8677ea897d1c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.903252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a7c6-397d-441c-825f-8677ea897d1c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.903534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4a7c6-397d-441c-825f-8677ea897d1c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.904935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 58/112] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:22.905856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6dbef-8e3a-45ba-9ba9-39aa3ea18056 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:22.906185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6dbef-8e3a-45ba-9ba9-39aa3ea18056 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:22.906484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6dbef-8e3a-45ba-9ba9-39aa3ea18056 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:22.907941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 59/113] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:22.908592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6dbef-8e3a-45ba-9ba9-39aa3ea18056 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:22.917036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.022404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 55/114] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:23.024715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.025240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.025392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6dbef-8e3a-45ba-9ba9-39aa3ea18056 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.025838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.045402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.049194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:23.050769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:23.064069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.064823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.065286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.065645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.066901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:23.068506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985996e9-f0fa-4a71-b9c0-eee66dae6ccf None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:23.071155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 60/115] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:22 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:23.072454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.072813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.073286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.092753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:23.094245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:23.140851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 56/116] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:23.142183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.184186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 61/117] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:23.185160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.192065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.266013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d572af-10ae-4cd3-9b58-b83cb9dfa7c3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.310010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.311835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.313232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.313889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.315951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:23.319631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:23.322744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 57/118] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:23.323800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.325237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d6ac4-bda2-4660-a988-b4ea07f60aae None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.327871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.362723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.363806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.364384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.364863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.368281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, user_id=2a01c135c7dc48f58bf2bfb169ad38a7, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:23.370654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:23.435643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8aa65-0626-4a5c-a945-dc3243b84860 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.473024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 62/119] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:23 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:23.473024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.480088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.480816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.481143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.481428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.495347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:23.501280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:23.502201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:23.502952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 58/120] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:23.503616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.566679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.579888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.617283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af485337-c43f-46fe-b9ee-f37c2297ada2 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.625223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201bb3-cdd2-4f25-bf7e-38d3e9ff6406 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.666539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.667185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.667574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.667964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.668779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=0edc6a0ed6ed40ff9505011e5dea4e37) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:23.668903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.669139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:23.669682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.669950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.670232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.670588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b7e99-5615-498e-9d40-92be283acb43 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:23.677256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 59/121] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/role_assignments?user.id=0edc6a0ed6ed40ff9505011e5dea4e37&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:23.678471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15e5-3637-4000-824e-4f33fba876c4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.678773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15e5-3637-4000-824e-4f33fba876c4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.679758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff15e5-3637-4000-824e-4f33fba876c4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.681494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 60/122] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:23.682326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff15e5-3637-4000-824e-4f33fba876c4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.685118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-f3280e38-0aee-4ee9-9067-720bd0435c28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:23.691999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:23.698921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 63/123] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:23.699732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.700421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.701679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.730193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:23.731577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:23.831324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff15e5-3637-4000-824e-4f33fba876c4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:23.846490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 64/124] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:23.847528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.885176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.902112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:23.902752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:23.903038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:23.903321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:23.904071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:23.904383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:23.905940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:23.923529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 61/125] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:23.924798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:23.943893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:23.998317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a175dfa-dbad-4488-9649-2301ca5b5838 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.037590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.038229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.038587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.038857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.039231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514a6fc-fcc6-450f-ad50-61906e582ea9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.039592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:24.039904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.041444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.050410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 65/126] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.056492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.078709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.080036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.080514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.081021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.093024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:24.098479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:24.099416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:24.100229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:24.102836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 62/127] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:24.103851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.157151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.171396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf089-32ea-4be6-9fd3-46aca435bd44 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.187052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.208721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.209407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.209701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.209983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.211040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc416c45-92b9-4d24-a4f7-c05f170b6249 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.211297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.212772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.215351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 66/128] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.220587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.248722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.249399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.249685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.249965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.251157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.252625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.254643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 63/129] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.255426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.356119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70148e50-a409-43b1-9905-088ededa3eb3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.401183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda4015-f95b-4084-b5c1-92562088301b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.413055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.414230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.414512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.414789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.415520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:24.415828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.417278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.426352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 67/130] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.427217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.437766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.453831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.454490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.454774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.455058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.456247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.459139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.462158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 64/131] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.465857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.510846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.621225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c6764-7a4a-4f98-bed3-3a6962dac08a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.666516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93187f-7551-4afc-9797-7d59aafbe824 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.691831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.692493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.692776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.694091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.696795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.698277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.705733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 68/132] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.706616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.735697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.736350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.736701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.736904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.739495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.740962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.744013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 65/133] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:24 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:24.744768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:24.821997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.884158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609f324-8d96-4455-9ffa-f972b26f0192 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.884743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39d7a-d981-41be-b326-e58b0a470b52 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:24.901357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:24.926946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.927721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.928013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.928299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.930072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:24.931057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:24.931266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, user_id=2a01c135c7dc48f58bf2bfb169ad38a7, role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.931687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:24.932135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/4c8c97aa5a724434b6ecf6c70f044b44/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:24.932738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:24.936057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, user_id=4c8c97aa5a724434b6ecf6c70f044b44, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:24.938220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a3a03-f3b2-4185-9a2a-0d4a0153e8dc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:25.015683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 66/134] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:24 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/2d3d839c7e5143618bb567643903898d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:25.021857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:25.043990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 69/135] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:24 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/4c8c97aa5a724434b6ecf6c70f044b44/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:25.045792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788d50e-d667-46c7-bf8c-d78346631705 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.046237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788d50e-d667-46c7-bf8c-d78346631705 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.046701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788d50e-d667-46c7-bf8c-d78346631705 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.051295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 70/136] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:25.057907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.058719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.059240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.110952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:25.118106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:25.233171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 71/137] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:25.260088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:25.322398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868a259-6bc3-4551-b914-c5fcfcdd19a7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:25.370799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:25.380428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74e6d0-e7f5-4116-9d1c-60178fc7694b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:25.382076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:25.402163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:25.402819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.403218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.403381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] PATH_INFO: `/v3/projects/baf2743cac19469aa446f2a30e8e7b26` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.404559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] RBAC: Authorizing `identity:get_project(project_id=baf2743cac19469aa446f2a30e8e7b26)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:25.406051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:25.410317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 67/138] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:25 2022] GET /identity/v3/projects/baf2743cac19469aa446f2a30e8e7b26 => generated 312 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:25.420074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:25.421742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:25.422403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.422684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.422961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.425051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:25.426511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:25.428275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 72/139] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:25.433801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:25.566199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21838f-87f2-487d-b475-c49e8ab7601a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:25.594145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470a9f-0e05-4c54-ab9d-76c176a0a79e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:25.633559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:25.634208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.634505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.634782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.635485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:25.637183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f853de-c8f1-4bcb-8ae8-a4a28357a575 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:25.653777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:25.654561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:25.654851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:25.655224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] PATH_INFO: `/v3/projects/baf2743cac19469aa446f2a30e8e7b26/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:25.657285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] RBAC: Authorizing `identity:create_grant(project_id=baf2743cac19469aa446f2a30e8e7b26, user_id=2a01c135c7dc48f58bf2bfb169ad38a7, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:25.658852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1765-3261-4a4b-96b6-4111f20e6d64 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:25.755806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 73/140] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:25 2022] POST /identity/v3/users => generated 283 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:16:25.791525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 68/141] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:25 2022] PUT /identity/v3/projects/baf2743cac19469aa446f2a30e8e7b26/users/2a01c135c7dc48f58bf2bfb169ad38a7/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:26.009314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52559a0f-8e2f-4690-ac14-598910c52ff1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.009657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52559a0f-8e2f-4690-ac14-598910c52ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.009848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52559a0f-8e2f-4690-ac14-598910c52ff1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.011419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 74/142] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:26.017254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.017556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.017826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.042085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:26.043323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:26.090665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d0bf7-3c81-4ad7-8261-a0d153d60e8c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.090955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d0bf7-3c81-4ad7-8261-a0d153d60e8c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.091220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d0bf7-3c81-4ad7-8261-a0d153d60e8c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.092650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 75/143] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:26.097176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.112847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.113235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.113684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.136182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:26.137767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:26.145437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 69/144] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:26.161462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.186764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.238005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 76/145] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:26.262422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.418976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6229d-49c6-48e2-807a-4f6459f76ff6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.441627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533c76-34b8-4d28-be8f-92dd55b39aeb None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.488262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:26.488572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:26.489278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.489555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.489828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.490122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.490597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.491080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.501991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:26.507366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:26.508224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:26.508982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 77/146] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:26.516986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:26.518870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:26.523155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:26.524011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:26.524771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:26.527316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 70/147] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:26.533433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.541046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.583005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.634893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89f7c-0e6a-45be-a8f2-5075b9ba1686 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.669663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b0c8-5431-4f04-a2c7-863e7124e2c1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.677707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:26.678336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.678621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.678954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.681686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:26.681686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:26.682526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:26.694908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 78/148] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:26.702637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.716405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:26.717433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.717919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.718418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.719418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:26.719925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:26.721713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:26.733838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 71/149] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:26.739759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:26.845331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.897406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:26.907296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3736583-6b30-4857-a8f7-92283e095dc7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.915940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b72b8-a6d0-4633-ac4b-1018181f8f55 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:26.965734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:26.966400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:26.966678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:26.966950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:26.968248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:26.969734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:26.971342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 72/150] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:26.977900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:27.011842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.012520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.012802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.013104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] PATH_INFO: `/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.014457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] RBAC: Authorizing `identity:get_user(user_id=0edc6a0ed6ed40ff9505011e5dea4e37)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.015935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.017644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 79/151] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37 => generated 289 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:27.023615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:27.103731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cdd4d-5194-4b4d-89cb-cc250b317844 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:27.140843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.141603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.141882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.142152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.189727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:27.219528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.222292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.225142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f95d-84db-458f-9a33-67d11c0d4892 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:27.279051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.279689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.279974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.280249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.285719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.287211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.289360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 80/152] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:27 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:27.294557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:27.316776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:27.345429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 73/153] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:27.351521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:27.436130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f1e52-f62c-403a-b991-be43176b4d65 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:27.445765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:27.483051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.483740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.484024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.484298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/0edc6a0ed6ed40ff9505011e5dea4e37/roles/9fc606fa235e4de0a1fa4bb9be08954e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.493837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, user_id=0edc6a0ed6ed40ff9505011e5dea4e37, role_id=9fc606fa235e4de0a1fa4bb9be08954e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.495331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d3d1-94e6-456e-83c2-482fd2ad9a0b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.496819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb1189-f85c-453f-a977-06762b469fd0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:27.538080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.560637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 74/154] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:27 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:27.561166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:27.615558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 81/155] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:27 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/users/0edc6a0ed6ed40ff9505011e5dea4e37/roles/9fc606fa235e4de0a1fa4bb9be08954e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:27.687555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4087-4370-4d2b-9d93-8d89adefa507 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:27.719005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:27.724814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:27.725542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:27.725957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:27.726239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:27.727063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:27.727368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:27.728915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fe295-d71f-445a-ae5c-f40daf1f8d92 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:27.735900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 75/156] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:27 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:28.759140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd754ed4-28a5-4079-aad5-8dcbd483d6b5 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:28.759658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd754ed4-28a5-4079-aad5-8dcbd483d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:28.759658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd754ed4-28a5-4079-aad5-8dcbd483d6b5 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:28.761364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 82/157] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:28.766362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaffad-6937-47ce-a619-7287449915f8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:28.766772 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaffad-6937-47ce-a619-7287449915f8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:28.766942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaffad-6937-47ce-a619-7287449915f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:28.789308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-abbaffad-6937-47ce-a619-7287449915f8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:28.790505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-abbaffad-6937-47ce-a619-7287449915f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:28.877691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 76/158] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:28.939061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd754ed4-28a5-4079-aad5-8dcbd483d6b5 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.001960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528478b-1826-4d1f-8d13-e0030fb34ba2 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.002287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528478b-1826-4d1f-8d13-e0030fb34ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.002512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528478b-1826-4d1f-8d13-e0030fb34ba2 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.003880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 77/159] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:29.009799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.010039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.010281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.034810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:29.036005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:29.052643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:29.098427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd754ed4-28a5-4079-aad5-8dcbd483d6b5 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.161580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.162344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.162624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.162897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.168199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 78/160] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:29.178607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-1d44d53c-2233-478a-83e7-9fb1823f11e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:29.184496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:29.185437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:29.186306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 83/161] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:29.191575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.201760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7e1e-88a6-46c6-b62b-d14019522527 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.202051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7e1e-88a6-46c6-b62b-d14019522527 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.202307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca7e1e-88a6-46c6-b62b-d14019522527 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.203667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 84/162] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:29.206260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca7e1e-88a6-46c6-b62b-d14019522527 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.286355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:29.366150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003adda-208b-4eab-a1a8-c0c2720841ad None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.389040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca7e1e-88a6-46c6-b62b-d14019522527 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.430456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.431096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.431372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.431644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.432371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:29.432674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:29.434362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4974-5919-48e2-9a4c-232413422ab6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:29.451894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 79/163] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:29.453226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.453483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.453760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.461054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.462087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.462584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.463091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.470862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:29.481040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:29.481040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:29.493919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:29.502027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:29.502963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:29.503738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 85/164] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:29.504508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.572308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 80/165] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:29.573796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.659929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:29.707886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf6ee-cb67-40a2-8f2a-f506ff3391cd None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.746475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.747187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.747469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.747750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] PATH_INFO: `/v3/users/4c8c97aa5a724434b6ecf6c70f044b44` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.748986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] RBAC: Authorizing `identity:get_user(user_id=4c8c97aa5a724434b6ecf6c70f044b44)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:29.751007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:29.752613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 86/166] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/users/4c8c97aa5a724434b6ecf6c70f044b44 => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:29.753542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.787760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb696f-2b68-41ed-95dd-425fa62b9837 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.826553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:29.867195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.867830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.868163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.868372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.869156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:29.869462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:29.870921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f16d9-991e-4840-91f1-49f6b272ea04 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:29.871039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187d508-6cea-4e5f-85d4-41263ae42bf3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:29.888131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 81/167] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:29.894004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53475854-587b-40ce-9eb8-b6edab915f82 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.894197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53475854-587b-40ce-9eb8-b6edab915f82 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.894450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53475854-587b-40ce-9eb8-b6edab915f82 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.895831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 82/168] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:29.898650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53475854-587b-40ce-9eb8-b6edab915f82 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:29.911084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:29.912258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.912752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.913923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.925974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-241ea554-de5f-4643-b823-990386d6fc77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:29.931958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:29.932843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:29.933922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:29.936108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 87/169] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:29.937479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:29.937901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:29.938411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:29.957700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:29.959145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:30.033868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 88/170] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:30.035413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.081080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53475854-587b-40ce-9eb8-b6edab915f82 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.096700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.099797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.159185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b6a2-e3a7-4687-b22a-ebf9ec3485cc None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.161894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.162537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.162814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.163146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.253726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.254380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.254658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.254930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.256137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.258158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.259766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 89/171] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:30.260917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.262401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.264860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.377123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 83/172] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:30.377913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.394256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.495781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a97619-220c-48ba-92a9-23f8b0af5a0d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.534494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.535356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.535735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.536126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.548087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:30.554645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:30.556009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:30.556787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 84/173] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:30.557509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.571251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa04ef9-fdd7-4a92-8443-d30c265e53d1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.593443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.608478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.609476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.609992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.610474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.611405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:30.611914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.613536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.622838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 90/174] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:30.624163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.638668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.672156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00b357-ad8e-4be9-aba5-3ab0ec8a6464 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.709578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.710227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.710509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.710782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.738206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d04da4-bbe8-464a-a65b-1c2028b9416e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.775118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.775773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.776037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.776309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.777524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.779398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f038-da71-48fd-b004-58f9a7afe095 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.780947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 91/175] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:30.781831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51c08-2c62-429c-ab14-2b4e78523c46 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.782112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51c08-2c62-429c-ab14-2b4e78523c46 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.782382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51c08-2c62-429c-ab14-2b4e78523c46 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.783595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 92/176] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:30.784296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51c08-2c62-429c-ab14-2b4e78523c46 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.788159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.789822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.842435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.857072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:30.890953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 85/177] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:30.891721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.903762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51c08-2c62-429c-ab14-2b4e78523c46 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:30.943808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:30.944433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.945293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.945738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.946463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:30.946771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:30.948164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c919471-5e7d-49a4-8053-fc288db9cbd5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:30.956306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 93/178] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:30.957509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a991-bc2f-43f0-963d-f8344fa77659 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:30.957794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a991-bc2f-43f0-963d-f8344fa77659 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:30.958072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a991-bc2f-43f0-963d-f8344fa77659 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:30.959296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 94/179] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:30.961144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a991-bc2f-43f0-963d-f8344fa77659 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:30.998434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd9916-e0ed-47d3-84ca-cde2cb032683 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.032507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.033250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.033533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.033808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] PATH_INFO: `/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.034996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] RBAC: Authorizing `identity:get_user(user_id=2a01c135c7dc48f58bf2bfb169ad38a7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.036450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd148b-8054-43e0-be88-17d20a745b48 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.038070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 86/180] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/users/2a01c135c7dc48f58bf2bfb169ad38a7 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.039278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace944-e8be-4751-9d27-d740a1ec2fff None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.039560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace944-e8be-4751-9d27-d740a1ec2fff None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.039835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eace944-e8be-4751-9d27-d740a1ec2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.056677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0eace944-e8be-4751-9d27-d740a1ec2fff None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:31.057886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0eace944-e8be-4751-9d27-d740a1ec2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:31.067044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a991-bc2f-43f0-963d-f8344fa77659 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.080403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.087600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.101717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.102610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.102736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.102918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.103722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=4c8c97aa5a724434b6ecf6c70f044b44) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:31.104031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.105513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.111386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 95/181] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:30 2022] GET /identity/v3/role_assignments?user.id=4c8c97aa5a724434b6ecf6c70f044b44&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.112001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.126233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 87/182] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:31.127837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfac04-94fa-4dce-b690-5495c48f66d7 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.128267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfac04-94fa-4dce-b690-5495c48f66d7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.128550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfac04-94fa-4dce-b690-5495c48f66d7 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.133058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 88/183] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:31.133058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfac04-94fa-4dce-b690-5495c48f66d7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.236003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2564663-6446-41a1-9341-13d617528f2e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.264794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfac04-94fa-4dce-b690-5495c48f66d7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.275191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.275855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.276140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.276428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.277660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.279127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b46a59-0467-472a-ab63-10176305cb3b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.280821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 96/184] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.282698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123754-1b04-43db-81e3-9dda6b832429 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.282698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123754-1b04-43db-81e3-9dda6b832429 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.282698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123754-1b04-43db-81e3-9dda6b832429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.301908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-19123754-1b04-43db-81e3-9dda6b832429 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:31.303235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-19123754-1b04-43db-81e3-9dda6b832429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:31.312867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.313573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.314475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.314475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.329171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.330641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.332430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 89/185] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.333305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.352251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.364436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.390771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 97/186] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:31.390771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.390771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.390771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.402521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:31.408339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:31.444100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e4301-5785-40f6-9a34-5604891bcf23 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.470945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 98/187] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:31.471960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.485693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.486316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.486600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.486876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.553443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.555657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.571008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cebcad-5c49-4a13-9690-7a7381d96fae None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.573291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.579827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.606203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.606843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.607123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.607398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.608539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.610030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.611646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 99/188] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.612404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.652896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 90/189] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:31.653770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.718604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef1a8-d5c9-4b23-ad87-8bb1440603e8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.752092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c81d3-f5cb-481d-8810-15b326fd82f0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.752705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.753342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.753618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.753882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.754670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:31.754974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.756401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.762236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 100/190] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=2d3d839c7e5143618bb567643903898d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:31.763025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:31.781990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.795848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.797448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.797736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.798013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.805284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:31.874840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:31.876529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:31.878439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70df54-7f85-423b-a436-3ee63a642bd3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:31.915669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:31.916589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:31.916589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:31.916856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:31.932940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:31.938646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:31.939544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:31.940335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 101/191] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:31.941182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.004295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 91/192] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:32.005179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.029454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.045761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.051303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad817ac-a247-4510-8465-f988d6cf0d63 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.084839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.086280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.086724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.087133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.097201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:32.103086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:32.104133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:32.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 102/193] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:32.106090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73387c6e-8574-4054-8bee-9cb7b3f6576c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.106161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.141234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.141846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.142123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.142396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.143104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.144583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.207066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cab8-3b9d-4421-be63-e17855e61020 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.239432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.240068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.240351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.240628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] PATH_INFO: `/v3/projects/baf2743cac19469aa446f2a30e8e7b26` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.241081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 92/194] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:32 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:32.242617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.242725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=baf2743cac19469aa446f2a30e8e7b26)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.244123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.245924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 103/195] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/projects/baf2743cac19469aa446f2a30e8e7b26 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.246725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.282735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.347244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.368926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186bdf-89a6-4314-a0d1-9311d45dea03 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.370208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307b5bd-7759-4161-a8bb-0ad1d54ad1d0 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.406094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.406602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.406892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.407681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.409817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:32.409952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.411367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.412155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.412920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.414624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.414624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.415828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.417827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.419444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 104/196] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.420221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.425387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 93/197] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.425387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.536606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.549875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8a4fc-6e82-4757-b94c-6589a7fb3572 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.560332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2160-de23-4346-820b-a4c3bbf185d3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.593344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.594276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.594874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.595005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.595766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:32.596135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.597808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.607566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 105/198] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.608485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.608700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.609628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.610049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.610463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.611403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=baf2743cac19469aa446f2a30e8e7b26, user.id=2a01c135c7dc48f58bf2bfb169ad38a7) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:32.611844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.613483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.621104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 94/199] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/role_assignments?user.id=2a01c135c7dc48f58bf2bfb169ad38a7&scope.project.id=baf2743cac19469aa446f2a30e8e7b26&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.622096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.631795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.720968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d39-c3d5-447d-aa7f-ee23963cfcfc None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.733921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d802d-fab8-423e-afaf-41446bd8ee48 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:32.759393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.760116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.760467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.760812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.761990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=49db8c9154be4ec9a9dda1840ba95849) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:32.761990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.763615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.769997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 106/200] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/role_assignments?user.id=49db8c9154be4ec9a9dda1840ba95849&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.771009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.774478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:32.775401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:32.775814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:32.776213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:32.777696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:32.779409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:32.781440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 95/201] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:32.790109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:32.800541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.956240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:32.980070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154435-efb2-4046-907d-9de156eccfa3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.027283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719e440-9a89-4374-8cd4-2868591d54af None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.067136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.068448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.069200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.070459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] PATH_INFO: `/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.072628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] RBAC: Authorizing `identity:get_user(user_id=0edc6a0ed6ed40ff9505011e5dea4e37)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.076148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:33.079260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 107/202] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/users/0edc6a0ed6ed40ff9505011e5dea4e37 => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:33.086429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:33.095478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.096112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.096392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.096665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.245859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.247344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:33.249597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:33.260522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b42d8-055b-4683-bc1b-c9bf06127207 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.298659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.299291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.299827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.300153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.378092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:33.437855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:33.481832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:33.503328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 96/203] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:33.509680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:33.515446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 108/204] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:33.521976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:33.665742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3bb34-293a-45dc-96a9-cd8030ba6995 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.702878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f5f0e-f668-4e9c-be02-17085a012cc7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.706174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.706825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.707100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.707373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.708627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.710665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:33.712360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 109/205] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:33 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:33.721799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:33.775000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.776072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.778066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.778540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.780750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.782438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:33.789243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 97/206] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:33 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:33.799383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:33.828157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:33.879986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:33.916789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e68ab9-7d7c-46b3-8151-29c8d5d6c19e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.958257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75361ac-1f78-4300-968b-43ba033c6d87 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:33.985460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:33.986080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:33.986356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:33.986629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:33.987425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, scope.project.id=719a4fb20cae4a80bbb5f3922c345311, user.id=0edc6a0ed6ed40ff9505011e5dea4e37) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:33.987730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:33.989200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250639f-7033-427d-97e6-4190a5290ce3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:34.004833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 110/207] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:33 2022] GET /identity/v3/role_assignments?user.id=0edc6a0ed6ed40ff9505011e5dea4e37&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:34.022978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:34.023950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:34.024231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:34.024508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:34.025340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=49db8c9154be4ec9a9dda1840ba95849) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:34.025651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:34.027201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb298e1-580f-41ca-891f-452b18460287 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:34.036471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 98/208] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:33 2022] GET /identity/v3/role_assignments?user.id=49db8c9154be4ec9a9dda1840ba95849&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=2d3d839c7e5143618bb567643903898d => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:34.727385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15954859-1c86-4383-b20a-6ba8b03622c3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:34.727715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15954859-1c86-4383-b20a-6ba8b03622c3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:34.727938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15954859-1c86-4383-b20a-6ba8b03622c3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:34.729381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 111/209] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:34.734759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:34.735045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:34.735325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:34.755516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:34.756700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:34.802037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bb3f-18a0-4129-b827-18de9e1af2e4 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:34.802352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bb3f-18a0-4129-b827-18de9e1af2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:34.802584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64bb3f-18a0-4129-b827-18de9e1af2e4 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:34.804034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 112/210] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:34.809369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:34.809709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:34.809983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:34.828626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:34.829848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:34.837062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 99/211] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:34.860004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:34.902052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:34.916118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 113/212] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:34.940345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:34.991654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b22f65-cb4a-48a1-b6ed-c76d42837ab1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:34.996974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.039310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.039970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.040254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.040565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.054423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c3e8-dfc1-4653-95d1-c4187ccfb7bd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.055657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:35.062308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:35.063196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:35.063967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:35.068948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 100/213] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:35.070523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.106982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.108004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.109230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.109658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.110995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.112590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.120090 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 114/214] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:35.121470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.178204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.185954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160f9c-856a-4fcb-9d1b-53574f24cafe None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.223624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.224354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.224804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.225114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.226595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:35.227064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.228590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6ac3f-65d5-4f18-aa85-3037fb4ccaa0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.231053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b5387-61d5-4138-8c8b-dc47c2611bda None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.237273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 101/215] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:35.242754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d585-d821-41a7-b41a-be5847a889ea None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.242948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d585-d821-41a7-b41a-be5847a889ea None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.243217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1d585-d821-41a7-b41a-be5847a889ea None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.244548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 102/216] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:35.249214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1d585-d821-41a7-b41a-be5847a889ea None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.253833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.271197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.271822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.272101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.272377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.273145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.274633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca48620-0e61-4966-a76e-109433e99680 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.358589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1d585-d821-41a7-b41a-be5847a889ea None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.387973 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 115/217] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:35 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:35.389587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.389873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.390190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.404231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.405333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.405832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.406338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] PATH_INFO: `/v3/users/fb4bd255a8194f959fc87cfa22d3aa96` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.417299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:35.418996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:35.423632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] RBAC: Authorizing `identity:get_user(user_id=fb4bd255a8194f959fc87cfa22d3aa96)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.425314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.427201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 103/218] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/users/fb4bd255a8194f959fc87cfa22d3aa96 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:35.433480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.457776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.512638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 116/219] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:35.534804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.560872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8e6c-148e-41ac-a4d6-a63be8a5623a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.597978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.598712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.598993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.599266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.647475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc2d8-6d7a-4e02-8418-479b919f251d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.650151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.690973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.692579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.700809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.701518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.701804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.702081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.716903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.719033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:35.727158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:35.728091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:35.728873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:35.729497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 117/220] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:35.737537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.822356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 104/221] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 14:16:35.830129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.865755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5329-34c1-4950-b2d2-38e872c4c694 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.902109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.903048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.903521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.903975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.904888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:35.905387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.906953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.917561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 118/222] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:35.923336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:35.943165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963c00-e8e4-46e6-b34d-0e2b190619d6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:35.952671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.964389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:35.984865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:35.985586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:35.985867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:35.986144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:35.987431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:35.988885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:35.990893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 105/223] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:35.996982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:36.035066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb006d-05e5-4e9b-8f65-4005ba3bcf7b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.071971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.073039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.073492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.073935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.075316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:36.076930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:36.079026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 119/224] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:36.084523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:36.105271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d503-89a0-4d6a-ae34-81519e4417fe None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.141906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.142538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.142824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.143498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.143942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=fb4bd255a8194f959fc87cfa22d3aa96) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:36.144262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:36.145757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:36.152301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 106/225] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:35 2022] GET /identity/v3/role_assignments?user.id=fb4bd255a8194f959fc87cfa22d3aa96&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:36.219680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7a07-3070-4a7e-b295-f9581dc52b94 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.256529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.257781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.258059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.258332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.259756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:36.261255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e69a-b21c-4234-967d-4081dbf8c31d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:36.263324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 120/226] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:36 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:36.265526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:36.268925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:36.283262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc65c-7966-4d1e-b8c2-7995e73b834e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.283592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc65c-7966-4d1e-b8c2-7995e73b834e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.283820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc65c-7966-4d1e-b8c2-7995e73b834e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.285573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 121/227] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:36.291189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.291467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.291750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.331497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:36.332757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:36.345384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:36.418718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e383124-f5f0-47ff-833b-b4488026ede4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.439658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 122/228] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:36.463264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.464341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.464854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.465394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/49db8c9154be4ec9a9dda1840ba95849/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.467582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, user_id=49db8c9154be4ec9a9dda1840ba95849, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:36.469348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:36.472605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:36.568492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 107/229] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:36 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/49db8c9154be4ec9a9dda1840ba95849/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:16:36.573415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:36.581106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae072f9-2f28-405e-b0f4-ab2950d9db30 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.631273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.631901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.632276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.632650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.653342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:36.658924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:36.659885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:36.661354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 123/230] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:36.666600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:36.809854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d6aa2-5145-46f0-9d4f-bae74497db32 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:36.848236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:36.855670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:36.856364 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:36.856650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:36.856959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:36.863212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:36.863212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:36.863212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:36.869238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 108/231] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:36.875331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:37.021308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215fa3-b6da-4a80-b78f-45dd2d1da812 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:37.068074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:37.068845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:37.069049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:37.069287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:37.071134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:37.072676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:37.074686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 124/232] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:36 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:37.079772 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:37.221894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92399d0f-4007-41e5-918f-698c6cac973b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:37.261184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:37.261837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:37.262126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:37.262406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:37.268336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:37.269834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f956f-0cd9-4ae4-bd99-d156d16d48c6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:37.271888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 109/233] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:37 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:37.277800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:37.309692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:37.400762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea563-0818-4065-85c0-320208cc6fd9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:37.443752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:37.444431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:37.445382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:37.445660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/49db8c9154be4ec9a9dda1840ba95849/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:37.447845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, user_id=49db8c9154be4ec9a9dda1840ba95849, role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:37.449321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb00c5f-59a1-4daa-b665-ea9962b31b01 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:37.529464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 125/234] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:37 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/49db8c9154be4ec9a9dda1840ba95849/roles/2d3d839c7e5143618bb567643903898d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:38.042211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97ae0-5f82-4da0-94b4-318b37f9fd9e None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.042494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97ae0-5f82-4da0-94b4-318b37f9fd9e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.043045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97ae0-5f82-4da0-94b4-318b37f9fd9e None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.044620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 110/235] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:38.049899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.050188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.050469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.078028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:38.079230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:38.173124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 126/236] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:38.208338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e97ae0-5f82-4da0-94b4-318b37f9fd9e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:38.270872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:38.358733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e97ae0-5f82-4da0-94b4-318b37f9fd9e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:38.403618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:38.404274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.404559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.404836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.416357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a30db246-25ce-4d4e-bd55-1542a12e5396 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:38.421279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:38.422345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:38.423345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 111/237] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:38.428513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:38.439480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:38.463332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc136f8-9856-4e82-be08-398d89fd9805 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.463618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc136f8-9856-4e82-be08-398d89fd9805 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.463888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc136f8-9856-4e82-be08-398d89fd9805 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.465426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 112/238] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:38.470713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.470993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.471269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.501294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:38.504571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:38.604384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308054-f20f-45cd-ac94-d679493f317f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:38.642067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 113/239] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:38.664656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:38.665306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.665744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.666174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.667057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:38.667496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:38.669024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:38.671534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:38.691186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 127/240] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:38.697366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:38.718971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:38.799844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:38.830562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2bef-12ea-41ff-84e4-8b476192303b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:38.832337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a4a-1431-425c-b110-e5b6cef80f1a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:38.871251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:38.872261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.872682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.872791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:38.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.873704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.874176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.874551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] PATH_INFO: `/v3/users/1d5f059f7e184b9590e06a62cf33a703` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.874998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:38.877082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12c231-3f06-4c95-8aef-c3376f474449 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:38.879731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 114/241] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:38.881133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8f39-7524-4f05-b414-e9d8022ee127 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.881583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8f39-7524-4f05-b414-e9d8022ee127 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.882126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad8f39-7524-4f05-b414-e9d8022ee127 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.883771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 115/242] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:38.885222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad8f39-7524-4f05-b414-e9d8022ee127 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:38.900358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=1d5f059f7e184b9590e06a62cf33a703)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:38.903545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401d74d-5246-4214-a784-2b67a25bd2d7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:38.905288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 128/243] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:38 2022] GET /identity/v3/users/1d5f059f7e184b9590e06a62cf33a703 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:38.907255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:38.907685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:38.908177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:38.929085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:38.930548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:39.002027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 129/244] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:39.003163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.005844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.015874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad8f39-7524-4f05-b414-e9d8022ee127 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.057308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.057929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.058202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.058470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.059415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.060857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8cb4f-2ffa-4417-86df-25b0817236a1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.065831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.107528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2f94e-a7d0-4f76-b37e-434772ea49ad None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.143593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.144215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.144505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.144780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.165993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 116/245] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:38 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:39.167545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f104d6-8a98-4bb5-aeb8-722b936bc269 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.168087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f104d6-8a98-4bb5-aeb8-722b936bc269 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.168559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f104d6-8a98-4bb5-aeb8-722b936bc269 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.170369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 117/246] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:39.171358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f104d6-8a98-4bb5-aeb8-722b936bc269 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.249692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.251522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.295918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f104d6-8a98-4bb5-aeb8-722b936bc269 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.339082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.342647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.343520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.343652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.343845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.362021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-89568a85-25ed-4b1c-850e-266520d731ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:39.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:39.369334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:39.370149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:39.378053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 118/247] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:39.378825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.379485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.379779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.391610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 130/248] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:39.392387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.400608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:39.401846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:39.410676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.487924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 119/249] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:39.489045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.528194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8902a-9572-4ba7-9dea-c7c9e6a37bb3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.580203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.580852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.581183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.581712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.583358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:39.583672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.585093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.597177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 131/250] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:39.598312 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.620075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609a8a4-9d30-494a-a96d-027ec6693f16 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.625386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.658923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.659615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.659894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.660169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.661590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.663000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.671670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 120/251] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:39.672275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.692814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.721038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b05a-7a51-41bd-a17e-89051a4c2768 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.761949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.762611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.762954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.763235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.775279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:39.780801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:39.781733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:39.782545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 132/252] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:39.783208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.792110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7ddd-8ba9-4b4c-b4b1-3bc356ff68df None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.829875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.830517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.830790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.831060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] PATH_INFO: `/v3/users/fb4bd255a8194f959fc87cfa22d3aa96` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.832234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] RBAC: Authorizing `identity:get_user(user_id=fb4bd255a8194f959fc87cfa22d3aa96)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.833826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.836801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 121/253] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/users/fb4bd255a8194f959fc87cfa22d3aa96 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:39.837735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.876889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.896565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354df9f-67c1-460e-a061-32b2675120ce None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.922373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:39.942291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.942929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.943209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.943483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.944289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=1d5f059f7e184b9590e06a62cf33a703) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:39.945205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.945567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd71c7-c385-4fb8-b31d-bb4d50472d2f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:39.947087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:39.961989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 133/254] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/role_assignments?user.id=1d5f059f7e184b9590e06a62cf33a703&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:39.963166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:39.989038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:39.989038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:39.989038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:39.989038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:39.989470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:39.989470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:39.989470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.003111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 122/255] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:40.004109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:40.118589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b0941-aacd-4853-96c8-dfd1f1b17fc9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:40.162717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:40.163679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.164311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.164846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.174216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:40.176132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.178497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 123/256] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:40 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:40.184773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:40.205382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:40.225585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3e538-a0d5-4535-9cd3-580cc51bdb6f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:40.271160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:40.271813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.272092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.272427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.275120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:40.276504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639a9dd-9f7f-4deb-a28d-b801190202ae None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.278387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 134/257] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:39 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:40.281231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659394a-a188-45ee-a5ce-cda9cf625073 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.281516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659394a-a188-45ee-a5ce-cda9cf625073 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.281791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659394a-a188-45ee-a5ce-cda9cf625073 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.285076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 135/258] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:40.285680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659394a-a188-45ee-a5ce-cda9cf625073 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:40.308304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:40.355897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920814b-179b-47e2-b7c0-2dc5d9116bff None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:40.396920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:40.397747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.398257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.398632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.478227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:40.479619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459790f-4dd9-4944-9992-418eea235887 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.509497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659394a-a188-45ee-a5ce-cda9cf625073 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:40.521388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:40.593373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:40.594102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.594387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.594662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/fb4bd255a8194f959fc87cfa22d3aa96/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.598486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, user_id=fb4bd255a8194f959fc87cfa22d3aa96, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:40.599974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.604680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 124/259] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:40.606039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.606332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.606648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.626341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:40.627723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:40.699203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 125/260] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:40.700211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:40.704257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 136/261] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:40 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/fb4bd255a8194f959fc87cfa22d3aa96/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:40.765365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:40.802915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:40.872227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c73e9-0219-4151-ac83-2a45adb3d7d6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:40.962004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:40.962758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:40.963037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:40.963308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:40.965114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:40.966604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:40.968283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 126/262] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:40 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:40.977368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:40.978945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:41.014292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd132c0-62c1-4bd9-8fa6-3725370fa6d1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:41.086459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:41.087045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:41.087329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:41.087603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:41.103018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:41.109975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:41.114323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:41.116014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 137/263] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:41.117670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc838-422d-4222-9c1d-9aba1a6275e3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:41.123001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:41.123594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:41.196248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:41.196884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:41.197188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:41.197466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:41.198275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=49db8c9154be4ec9a9dda1840ba95849) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:41.198581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:41.200040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:41.223201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 127/264] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:40 2022] GET /identity/v3/role_assignments?user.id=49db8c9154be4ec9a9dda1840ba95849&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:41.339480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f8266-a4fd-4204-ae85-7329a070681d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:41.393601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:41.400061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:41.400921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:41.401337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:41.401812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:41.403196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:41.403725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:41.406430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:41.422077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 138/265] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:41.429918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:41.590846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45955589-3dfa-48d9-96af-3203359b30fb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:41.653142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:41.653802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:41.654080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:41.654351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] PATH_INFO: `/v3/users/49db8c9154be4ec9a9dda1840ba95849` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:41.657769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] RBAC: Authorizing `identity:get_user(user_id=49db8c9154be4ec9a9dda1840ba95849)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:41.659258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:41.660939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 128/266] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:41 2022] GET /identity/v3/users/49db8c9154be4ec9a9dda1840ba95849 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:41.672407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:41.846259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2e26-7694-417f-aa8d-d85b6aab7a4e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:41.898313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:41.898954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:41.899242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:41.899529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:41.981838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:41.989671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:41.991171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3966999-98c6-4648-8826-b81c3f0ce4b5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:42.128191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 139/267] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:42.133935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.160537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fab2-a23e-46b0-9f50-3678d081b319 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.160817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fab2-a23e-46b0-9f50-3678d081b319 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.161117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2fab2-a23e-46b0-9f50-3678d081b319 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.162487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 140/268] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:42.171336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab67d-5a64-49f2-be60-8c09f761a6b0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.171601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab67d-5a64-49f2-be60-8c09f761a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.171879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eab67d-5a64-49f2-be60-8c09f761a6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.201624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d5eab67d-5a64-49f2-be60-8c09f761a6b0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:42.202814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d5eab67d-5a64-49f2-be60-8c09f761a6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:42.233361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.326807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436e07-50d2-4097-84f4-f74d4b80ace3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:42.338630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 141/269] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 29 14:16:42.345433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833863-03ce-4f8b-bca1-35dd189a461e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.346395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833863-03ce-4f8b-bca1-35dd189a461e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.346860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833863-03ce-4f8b-bca1-35dd189a461e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.348638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 142/270] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:42.356963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.357497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.357976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.377579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:42.379130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:42.383307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:42.383976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.384260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.384539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.385762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:42.387207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:42.394116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 129/271] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:42.394116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.404380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.465170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 143/272] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:42.466045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.516239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3660a45-e997-4643-9a9a-6daa70f6af38 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:42.524624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.553771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:42.555240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.555518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.555795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.570912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:42.577734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:42.578614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:42.579410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 130/273] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:42.580806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.582276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7ac90-8ba2-487d-92af-af569deddf45 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:42.621322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:42.621956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.622237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.622510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.623299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=49db8c9154be4ec9a9dda1840ba95849) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:42.623820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:42.625073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9448699-86f5-4c15-ac93-df0038304338 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 144/274] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/role_assignments?user.id=49db8c9154be4ec9a9dda1840ba95849&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=2d3d839c7e5143618bb567643903898d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb2565-c0e1-46ef-9f54-737254d27071 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb2565-c0e1-46ef-9f54-737254d27071 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb2565-c0e1-46ef-9f54-737254d27071 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 145/275] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:42.641129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb2565-c0e1-46ef-9f54-737254d27071 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.670299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.713356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d15eb-14c0-460f-b740-edb976f6049b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:42.756810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:42.758198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.758545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.758888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.771458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-2a67172d-3247-4b31-bde3-88e29ad3088c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:42.778225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:42.779179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:42.780074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 131/276] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:42.784763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.784763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.784763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.792422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb2565-c0e1-46ef-9f54-737254d27071 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:42.802645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:42.803462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:42.828291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.839512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:42.840234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:42.840585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:42.840933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:42.841751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:42.842061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:42.843442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:42.858756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 146/277] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:42.858756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.895885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 132/278] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:42.897094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:42.955641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:42.974986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8085-6fb5-4b23-ad58-8c1b3dac2430 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.011765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.012384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.012665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.013048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.013677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:43.013979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.015544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.018532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc1cd4-4ac8-4ec6-a587-55bcaedc2871 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.024155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 147/279] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.024981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.057798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.058453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.058998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.059276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] PATH_INFO: `/v3/users/b5465413cb9c49939980e20d8af80eac` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.075474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] RBAC: Authorizing `identity:get_user(user_id=b5465413cb9c49939980e20d8af80eac)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.076959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.083333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 133/280] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:42 2022] GET /identity/v3/users/b5465413cb9c49939980e20d8af80eac => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.083333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.098697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.139431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640fa2f-c6aa-4427-a511-330730abd53d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.180748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.181437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.182052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.182332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.183927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.186010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.192635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 148/281] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.192635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.193344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944bb48-9742-4c93-b438-6c493ea75033 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.204800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.232917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.233594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.233874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.234149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] PATH_INFO: `/v3/users/1d5f059f7e184b9590e06a62cf33a703` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.235517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] RBAC: Authorizing `identity:get_user(user_id=1d5f059f7e184b9590e06a62cf33a703)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.237141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.238907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 134/282] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/users/1d5f059f7e184b9590e06a62cf33a703 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.244072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.307022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88a68f-b1f3-4a16-a14f-4881df4322c8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.327789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.344192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.344819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.345134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.345415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.346216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, user.id=b5465413cb9c49939980e20d8af80eac, scope.system=all) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:43.346758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.346833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0324c-b2e2-43e3-8f35-74ca7d177f24 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.347955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.357327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 149/283] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/role_assignments?user.id=b5465413cb9c49939980e20d8af80eac&scope.system=all&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.358632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.388825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.389921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.390402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.390874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.391783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.393725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.460726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.488640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca7601-d1db-4113-83a8-27299b921441 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.509645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 135/284] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:16:43 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:43.544443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120924-80ea-478b-ad75-88c299608721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.545208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120924-80ea-478b-ad75-88c299608721 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.545488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120924-80ea-478b-ad75-88c299608721 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.545761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120924-80ea-478b-ad75-88c299608721 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.547002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120924-80ea-478b-ad75-88c299608721 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.548452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120924-80ea-478b-ad75-88c299608721 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.550517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 150/285] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:43.556094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.563453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646643b7-036b-4b69-ba98-e1c3ad10b2f4 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.563768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646643b7-036b-4b69-ba98-e1c3ad10b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.564015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646643b7-036b-4b69-ba98-e1c3ad10b2f4 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.565617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 151/286] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:43.571738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.572045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.572299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.594038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:43.595300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:43.678184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 152/287] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:43.696591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efbf0e-6aab-4420-90a8-91234efab926 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.702439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:43.739972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.740577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.740861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.741193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/1d5f059f7e184b9590e06a62cf33a703/roles/9fc606fa235e4de0a1fa4bb9be08954e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.744088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, user_id=1d5f059f7e184b9590e06a62cf33a703, role_id=9fc606fa235e4de0a1fa4bb9be08954e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:43.745604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:43.787329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.820638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c2450-7180-4740-b2dc-b832372c7dd8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:43.828123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:43.837417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 136/288] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:16:43 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/users/1d5f059f7e184b9590e06a62cf33a703/roles/9fc606fa235e4de0a1fa4bb9be08954e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:43.866170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:43.867029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:43.867405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:43.867685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:43.881196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:43.887198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:43.888237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:43.889515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 153/289] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:43.894611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:44.019742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0393a-456e-4034-924c-cd022a9c48e9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:44.056388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:44.057040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:44.057329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:44.057609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:44.058345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:44.058905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:44.060305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:44.069410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 137/290] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:44.075563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:44.203813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ca82-1eca-428a-85a2-0b32538e16da None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:44.248068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:44.253710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:44.254363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:44.254641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:44.254914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] PATH_INFO: `/v3/users/fb4bd255a8194f959fc87cfa22d3aa96` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:44.261715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] RBAC: Authorizing `identity:get_user(user_id=fb4bd255a8194f959fc87cfa22d3aa96)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:44.263243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:44.265245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 154/291] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:44 2022] GET /identity/v3/users/fb4bd255a8194f959fc87cfa22d3aa96 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:44.270540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:44.360785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:44.421320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06924-92c7-4044-b7a9-cf6c67814a03 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:44.468417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:44.469063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:44.469350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:44.469629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:44.552692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:44.554736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:44.641360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:44.692775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 138/292] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:44.698088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:44.857668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb385-351d-4c20-8de8-cbde5e836558 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:44.907553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:44.908203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:44.908483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:44.908764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:44.909953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:44.911468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801d4c-947f-48c2-89b1-f9d6db0f8f9f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:44.914033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 155/293] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:44 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:44.919627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:45.070888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59fa8a-8d59-4cb2-89eb-f01e147d26ac None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:45.122929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:45.123910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:45.124396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:45.124859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:45.125860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=fb4bd255a8194f959fc87cfa22d3aa96) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:45.126372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:45.128001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c1c8f-a06b-4c59-8f1a-dce87632942c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:45.141687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 139/294] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:44 2022] GET /identity/v3/role_assignments?user.id=fb4bd255a8194f959fc87cfa22d3aa96&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:45.662199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39deb338-664c-4268-9eb7-3aaf4c5e73b1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:45.662567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39deb338-664c-4268-9eb7-3aaf4c5e73b1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:45.662766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39deb338-664c-4268-9eb7-3aaf4c5e73b1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:45.665099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 156/295] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:45.670207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:45.670655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:45.670824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:45.692048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:45.693345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:45.714689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:45.773521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 140/296] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:45.802040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39deb338-664c-4268-9eb7-3aaf4c5e73b1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:45.872037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:45.909252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39deb338-664c-4268-9eb7-3aaf4c5e73b1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:45.944543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:45.945897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:45.946175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:45.946448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:45.957225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-d9d5e3de-a6fb-4887-aa72-e03a6e4aefab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:45.962977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:45.963832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:45.964627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 157/297] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:45.971378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.049499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2671-ca4b-4fc4-a92c-c15f7bcfb101 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.049855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2671-ca4b-4fc4-a92c-c15f7bcfb101 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.050062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2671-ca4b-4fc4-a92c-c15f7bcfb101 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.051693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 158/298] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:46.057249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a958-51e6-4ba3-8242-e0f0985f0fc7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.057533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a958-51e6-4ba3-8242-e0f0985f0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.057809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a958-51e6-4ba3-8242-e0f0985f0fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.077321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3d76a958-51e6-4ba3-8242-e0f0985f0fc7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:46.078290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4001d-cf53-47ad-b3a9-bc05a7ea8cf0 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.078514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3d76a958-51e6-4ba3-8242-e0f0985f0fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:46.118462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.119095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.119374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.119650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.120377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:46.120684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.122116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.134267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 141/299] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:46.140003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.160908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.177150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 159/300] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:46.178280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c304d5-422a-46aa-a910-fe8e91c9eb13 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.178575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c304d5-422a-46aa-a910-fe8e91c9eb13 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.178877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c304d5-422a-46aa-a910-fe8e91c9eb13 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.180448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 160/301] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:46.186097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.186384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.186665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.206035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:46.207133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:46.254690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334527c-f904-419f-abae-f40360a1e372 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.265475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.287863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.289290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.289518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.289731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 161/302] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:46.289829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] PATH_INFO: `/v3/users/b5465413cb9c49939980e20d8af80eac` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.290962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] RBAC: Authorizing `identity:get_user(user_id=b5465413cb9c49939980e20d8af80eac)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.291297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.292557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.294175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 142/303] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/users/b5465413cb9c49939980e20d8af80eac => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:46.299931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.392829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.403576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d618b25-2039-476a-a3ba-4bcf59859ef2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.409361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282e79-b1ef-443e-8989-8c583af57443 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.440299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.440951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.441264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.441540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.443299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.445348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.446941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 162/304] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:46.448059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.448906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.449380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.449817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] PATH_INFO: `/v3/system/users/b5465413cb9c49939980e20d8af80eac/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.451803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5465413cb9c49939980e20d8af80eac, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.452204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.453707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.508118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.551254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bf13-b3eb-4eab-8728-0f8365569bec None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.556660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 143/305] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Jul 29 14:16:46 2022] PUT /identity/v3/system/users/b5465413cb9c49939980e20d8af80eac/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:16:46.560964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.591069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.591744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.592089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.592323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.604860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-65d3aaa7-837b-4d86-aa98-9c691f0f4251 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:46.611017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:46.612024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:46.612837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 163/306] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:46.614074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774d4ed-655f-4f05-86b2-40925483ceb1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.614074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774d4ed-655f-4f05-86b2-40925483ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.614198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774d4ed-655f-4f05-86b2-40925483ceb1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.615706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 164/307] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:46.620747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774d4ed-655f-4f05-86b2-40925483ceb1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.692434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492314d-7c12-494c-ad6d-d8f56a4c01bb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.735351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774d4ed-655f-4f05-86b2-40925483ceb1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:46.738406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.738707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.738997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.739273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] PATH_INFO: `/v3/groups` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.739945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.741461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.752769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.770807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:46.771427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.771707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.771997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.772746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:46.773595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:46.774956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81ca3-b632-455e-8f19-96a294c63484 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:46.783554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 165/308] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:46.784706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:46.785011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:46.785300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:46.802417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:46.803596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:46.819146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 144/309] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Jul 29 14:16:46 2022] POST /identity/v3/groups => generated 229 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:16:46.820854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:46.841895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:46.890339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 166/310] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:46.914039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.001870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ff577-66d8-4ad0-88e2-e59b908fbd7f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.043686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.044601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.045045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.045478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] PATH_INFO: `/v3/users/79ba2d6605464b87b8e9565502f80ce3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.062095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908301f9-35be-480c-b952-42fd86b77c9c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.062941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] RBAC: Authorizing `identity:get_user(user_id=79ba2d6605464b87b8e9565502f80ce3)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:47.064408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:47.066369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 145/311] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/users/79ba2d6605464b87b8e9565502f80ce3 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:47.072855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.077528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:47.105106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.105896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.106755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.107017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.119323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:47.124704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:47.125843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:47.126829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:47.128809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 167/312] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:47.134589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.167021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:47.222088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69fb01-fa39-4e5f-9dc6-d6e1b5ecd7af None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.260020 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.260802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.261211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.261588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.262494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, user.id=79ba2d6605464b87b8e9565502f80ce3, scope.system=all) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:47.262907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:47.264467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:47.270472 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 146/313] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 14:16:47 2022] GET /identity/v3/role_assignments?user.id=79ba2d6605464b87b8e9565502f80ce3&scope.system=all&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:47.274915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa478be5-7cb9-4c28-a0a7-c1d1e1af0129 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.321113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.321618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.321900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.322181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.322939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:47.323239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:47.324671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:47.335203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 168/314] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:47.342607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.457195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c843-bcd6-4356-a3fb-e30c13bd6060 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.487526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:47.493900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.494776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.495128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.495475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] PATH_INFO: `/v3/users/1d5f059f7e184b9590e06a62cf33a703` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.496732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] RBAC: Authorizing `identity:get_user(user_id=1d5f059f7e184b9590e06a62cf33a703)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:47.498372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:47.500214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 147/315] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:47 2022] GET /identity/v3/users/1d5f059f7e184b9590e06a62cf33a703 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:47.505900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.540657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:47.617555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f243d-6819-4599-a48c-7cb7d9c0ecf2 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.654589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.655368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.655771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.656152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.738081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:47.739828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:47.775668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:47.850088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 169/316] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 14:16:47.856488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:47.959304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2a5e-9b00-4e48-bb65-21d1c5e2bdd1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:47.996417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:47.997235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:47.997563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:47.997842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:47.999033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:48.000484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235402-2f79-4ae1-afed-9ccea56a70e5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:48.007264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 148/317] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:47 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:48.008809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:48.119784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ee51d-8abd-4bbe-b2a1-ba93c0fde720 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:48.150705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:48.165520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:48.166402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.166996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.167482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:48.168482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, scope.project.id=e38b7fd07d644392bc554b82457fcc43, user.id=1d5f059f7e184b9590e06a62cf33a703) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:48.169011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:48.170657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12f68d-4f86-47f3-bdf6-e1cd8df69b9d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:48.177564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 170/318] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:16:48 2022] GET /identity/v3/role_assignments?user.id=1d5f059f7e184b9590e06a62cf33a703&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:48.591538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbee3a-1614-4342-b3b6-88379d0114e6 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.591911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbee3a-1614-4342-b3b6-88379d0114e6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.592089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbee3a-1614-4342-b3b6-88379d0114e6 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:48.593707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 149/319] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:48.598662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.598953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.599254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:48.618601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:48.619781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:48.691660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 171/320] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:48.714536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbee3a-1614-4342-b3b6-88379d0114e6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:48.784835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:48.820653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbee3a-1614-4342-b3b6-88379d0114e6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:48.856393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:48.856959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.857541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.857615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:48.868077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e79ce8c5-2165-4ca7-abe8-2e611eb7009c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:48.873247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:48.874122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:48.874906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 150/321] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:48.881392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:48.986073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9bea-3376-4b86-b1b3-dd2bc9b1389f None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.986390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9bea-3376-4b86-b1b3-dd2bc9b1389f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.986634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9bea-3376-4b86-b1b3-dd2bc9b1389f None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:48.988139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 151/322] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:48.989652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669bde2-021a-43ff-b247-b9011870da23 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:48.992824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:48.993155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:48.993432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.011648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:49.012856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:49.016441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.027637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.028261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.029947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.029947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.029947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:49.029947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.031300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.039626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 172/323] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:49.044872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.086388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 152/324] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:49.108012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.149301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41048d5-2f72-4202-bfae-cfa7a74edd3e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.174255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.186047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.186693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.186971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.187246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] PATH_INFO: `/v3/users/b5465413cb9c49939980e20d8af80eac` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.188427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] RBAC: Authorizing `identity:get_user(user_id=b5465413cb9c49939980e20d8af80eac)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.189893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.191664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 173/325] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/users/b5465413cb9c49939980e20d8af80eac => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:49.197950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.222785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04068009-c04b-4f4f-aac8-86202e5cde97 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.256158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.260032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.260681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.260960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.261266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.262406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.263852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.265576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 153/326] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:49.270858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.307036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a92d4-d1a0-4cf5-997a-feca22365c21 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.341442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.342061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.342336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.342610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.343398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] RBAC: Adding query filter params (role.id=680f93d19a99495a81d0cf39e7ac3ce5, user.id=b5465413cb9c49939980e20d8af80eac, scope.system=all) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:49.343707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.345160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca8d9b-d1ee-4bab-aa20-aee3399137be None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.356746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 174/327] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/role_assignments?user.id=b5465413cb9c49939980e20d8af80eac&scope.system=all&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:49.358174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c7e-6741-42fc-993f-6e4a8a599620 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.358407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c7e-6741-42fc-993f-6e4a8a599620 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.358717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c7e-6741-42fc-993f-6e4a8a599620 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.359974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 175/328] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:49.365867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.366206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.366985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.378457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cc864-ed74-45a0-a314-5e617d199721 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.387351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:49.388588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:49.407898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.419722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.420351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.420628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.420891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] PATH_INFO: `/v3/groups` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.421607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.423089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.470038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 176/329] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:49.494050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.529779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 154/330] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Jul 29 14:16:49 2022] POST /identity/v3/groups => generated 226 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:16:49.541932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.608277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079668e1-3db5-4ad8-9a5d-a7c5ce56e869 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.645205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.645862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.646134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.646402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.658103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:49.663286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:49.664773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:49.665845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 177/331] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:49.671292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.776495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cba02-031e-4ce6-ae23-27deeef2e8fc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.812409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.813073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.813359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.813636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.814366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:49.814673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.816043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.825334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 155/332] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:49.830902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:49.941422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f5130-ae15-4d68-a0e8-b78f55a7f9af None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:49.943166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:49.981720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:49.982363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:49.982638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:49.982909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] PATH_INFO: `/v3/users/79ba2d6605464b87b8e9565502f80ce3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:49.984136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] RBAC: Authorizing `identity:get_user(user_id=79ba2d6605464b87b8e9565502f80ce3)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:49.985625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3448355-9594-42fb-a8ca-8e1e9a1bee10 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:49.987523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 178/333] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:49 2022] GET /identity/v3/users/79ba2d6605464b87b8e9565502f80ce3 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:49.993920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:50.052756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:50.116453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81068e90-c4e0-4993-b9f4-9c209689a49c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:50.159816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:50.159816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:50.159816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:50.160184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] PATH_INFO: `/v3/system/users/79ba2d6605464b87b8e9565502f80ce3/roles/9fc606fa235e4de0a1fa4bb9be08954e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:50.161479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=79ba2d6605464b87b8e9565502f80ce3, role_id=9fc606fa235e4de0a1fa4bb9be08954e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:50.162865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f588a4-5f73-4522-ad7c-35d21095f835 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:50.238853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 156/334] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Jul 29 14:16:49 2022] PUT /identity/v3/system/users/79ba2d6605464b87b8e9565502f80ce3/roles/9fc606fa235e4de0a1fa4bb9be08954e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:51.941799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1bb6-e094-4fa5-9251-237a1d4a52f0 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:51.942300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1bb6-e094-4fa5-9251-237a1d4a52f0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:51.942406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c1bb6-e094-4fa5-9251-237a1d4a52f0 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:51.943993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 179/335] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:51.950003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0f93-beac-4fa7-827b-ccd7cf3740d9 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:51.950465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0f93-beac-4fa7-827b-ccd7cf3740d9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:51.950540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0f93-beac-4fa7-827b-ccd7cf3740d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:51.974202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-978b0f93-beac-4fa7-827b-ccd7cf3740d9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:51.975438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-978b0f93-beac-4fa7-827b-ccd7cf3740d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:52.059933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 157/336] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:52.102282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c1bb6-e094-4fa5-9251-237a1d4a52f0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:52.223587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c1bb6-e094-4fa5-9251-237a1d4a52f0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:52.259749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:52.260405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.260691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.260966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.261250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5ee1-a532-4b47-a70c-414b5f0c787e None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.261795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5ee1-a532-4b47-a70c-414b5f0c787e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.262271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5ee1-a532-4b47-a70c-414b5f0c787e None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.264080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 158/337] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:52.269308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.269826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.270320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.272127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:52.279435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:52.280333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:52.281151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 180/338] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:52.286694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:52.312915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:52.314134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:52.348569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:52.399587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:52.486055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 159/339] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:52.497008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a56e29-018f-41d2-87cd-7901650a53b9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:52.521276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:52.571976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:52.572603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.572881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.573217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.573945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:52.574294 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:52.575680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:52.606760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 181/340] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:52.607775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:52.727957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977f3c-8304-452d-b9cb-3be4e9c42078 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:52.785501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:52.800031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e53e8c-bf52-40b6-808c-c09720e8f0df None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:52.822391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:52.823060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.823340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.823610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.853978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:52.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:52.859839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:52.860626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 160/341] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:52.861320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:52.861884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:52.861979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:52.864740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:52.868781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:52.954492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:52.955478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:52.958185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4817b-a969-4311-aee9-1e8735792b3b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.059139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:53.070846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 182/342] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:53.075981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e9f3-1442-4aee-b3a3-87888c0c589c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.076529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e9f3-1442-4aee-b3a3-87888c0c589c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.077025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e9f3-1442-4aee-b3a3-87888c0c589c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.078584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 183/343] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:53.080708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e9f3-1442-4aee-b3a3-87888c0c589c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:53.118502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31495c2a-12b2-4fac-8d05-f8442732c1dc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.196823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.197484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.197760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.198032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.198761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:53.199063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:53.201984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d642a-378f-4374-8764-6ae12e1b1ddf None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.223280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 161/344] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:53.224569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.224943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.229645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:53.263350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:53.287880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e9f3-1442-4aee-b3a3-87888c0c589c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.332491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.333103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.333383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.333655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.336698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:53.338496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.340677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 184/345] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:53.341616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:53.421400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 162/346] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:53.423009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:53.455453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:53.461342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:53.507430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e34b38-6a77-4775-813f-014597e00d25 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.562367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.563116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.563483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.563837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.568739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d18866-fb8b-4062-bd3c-c361185147d3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.612152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.612718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.613021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.613286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.620664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:53.622451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82957-ab59-4005-b53a-d2449b943f54 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.624422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 163/347] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:53.626060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714af71-4c34-444a-881c-d9a4e02ff21a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.626060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714af71-4c34-444a-881c-d9a4e02ff21a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.626230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714af71-4c34-444a-881c-d9a4e02ff21a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.627421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 164/348] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:53.628109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714af71-4c34-444a-881c-d9a4e02ff21a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:53.664964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:53.666833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb00be-1ada-4eb0-a215-eac3f581ab03 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.737517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:53.744730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:53.763018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714af71-4c34-444a-881c-d9a4e02ff21a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.805637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 185/349] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:53.807288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd726b8-d178-4463-aea9-efc7b27f176a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.812274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd726b8-d178-4463-aea9-efc7b27f176a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.812404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.812404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.812404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.812404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.812687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd726b8-d178-4463-aea9-efc7b27f176a None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.812687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 186/350] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:53.812958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd726b8-d178-4463-aea9-efc7b27f176a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:53.824971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-adb23c39-3665-4186-849a-e6b7173fd1b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:53.830422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:53.831341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:53.832151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 165/351] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:53.837163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcc598-5a7b-4ca9-831a-f8e2a0e085c6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.837163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcc598-5a7b-4ca9-831a-f8e2a0e085c6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.837163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcc598-5a7b-4ca9-831a-f8e2a0e085c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.856812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8bbcc598-5a7b-4ca9-831a-f8e2a0e085c6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:53.858035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8bbcc598-5a7b-4ca9-831a-f8e2a0e085c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:53.938072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 166/352] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:53.939195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd726b8-d178-4463-aea9-efc7b27f176a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:53.939319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.939521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.939798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.959546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:53.961813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:53.987338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:53.988065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:53.988419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:53.988791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:53.989975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=719a4fb20cae4a80bbb5f3922c345311) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:53.990253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:53.991726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:53.998914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 187/353] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=2d3d839c7e5143618bb567643903898d => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:53.999746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.032217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.048970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 167/354] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:54.049738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.075519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.132029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f2752-6cff-4550-8878-beeec91f5ace None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.182117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.182778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.183058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.183445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.184899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.186378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf7119-3243-49d3-9414-bee3365226dd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.187030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed192bc-d403-49ab-8f92-3ad445e7dfb3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.188417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 188/355] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:53 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:54.190483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7d1d-4fac-4f56-8b2b-b3df27c6a691 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.190483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7d1d-4fac-4f56-8b2b-b3df27c6a691 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.190483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad7d1d-4fac-4f56-8b2b-b3df27c6a691 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.191819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 189/356] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:54.192528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7d1d-4fac-4f56-8b2b-b3df27c6a691 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.234939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.235585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.235870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.236152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.236886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:54.237245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.238627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.249973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 168/357] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:54.250840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.316379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.327776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad7d1d-4fac-4f56-8b2b-b3df27c6a691 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.334498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.375272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.375917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.376200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.376474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.384035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd475e2-2ddf-4448-8cb6-dec244a75a00 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.393677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:54.399489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:54.400387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 29 14:16:54.401212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:54.402721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 190/358] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:54.403028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.432546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.433026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.433335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.433611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.447808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-7ef84ef3-f1cd-4827-b40a-4ef13747522e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:54.454081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:54.455005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:54.455867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:54.458498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 169/359] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:54.460208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.460717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.461161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.485360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:54.486771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:54.535060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdcfe-2fc8-4eb9-958f-137cf31494cb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.579393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.580050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.580326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.580601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.581826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.583296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.584936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 191/360] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:54.587702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 170/361] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:54.588415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.589070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.624697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.666203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.722395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a8be8-87d8-4941-9787-4484df90aa53 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.736055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80e1cf-16fa-4fe2-9f7b-f3dd7eab8aa8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.775719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.776373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.776650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.776928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.777788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:54.779617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.779740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.780962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.781903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.782185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.782459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.791143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 171/362] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:54.794726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:54.864687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.866968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.891115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.917595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628618e1-f860-4a40-8f7e-634521a6cac4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:54.943568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:54.959988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:54.961322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:54.961738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:54.962150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:54.962884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:54.963188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:54.964572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:54.974742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 172/363] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:54.975324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.006769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 192/364] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:55.008266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.089781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1f99b-b174-482a-929a-48d313b8957b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.137288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.137912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.138188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.138456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.139254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=719a4fb20cae4a80bbb5f3922c345311) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:55.139560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.140982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.146175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b21a2-1124-4922-ac83-57f983aded4e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.146654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 173/365] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:16:54 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.147453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.180844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.199605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.205321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.205321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.205321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.216013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:55.221184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:55.222115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:55.223006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 193/366] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:55.223670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.229300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.315273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd71b3-2384-4ef5-8cf4-8aeb173f6c89 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.361885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.362579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.362856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.363179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] PATH_INFO: `/v3/users/79ba2d6605464b87b8e9565502f80ce3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.364360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] RBAC: Authorizing `identity:get_user(user_id=79ba2d6605464b87b8e9565502f80ce3)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.365828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.366208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253b92-2f31-4cdd-9db3-a3017f18be6a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.367873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 174/367] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/users/79ba2d6605464b87b8e9565502f80ce3 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.368650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.403111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.403735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.404016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.404351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.464322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.482249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.483773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.501834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c634-a43c-4159-b13e-eb827179e475 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.519238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.545796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.546455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.546780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.547257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.548647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.551032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.554378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 175/368] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.555677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.611806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 194/369] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:55.612651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.704123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b744b-d040-452b-b394-aa5649a9e044 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.719513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.742284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee096fb-f819-40da-8a5d-5feb87fc90d3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.750099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.750727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.751051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.751317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.753177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:55.753497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.755580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.767448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 176/370] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.768369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.798366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.799417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.799696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.799970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.800774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] RBAC: Adding query filter params (role.id=9fc606fa235e4de0a1fa4bb9be08954e, user.id=79ba2d6605464b87b8e9565502f80ce3, scope.system=all) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:55.801151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.803233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.820217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 195/371] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/role_assignments?user.id=79ba2d6605464b87b8e9565502f80ce3&scope.system=all&role.id=9fc606fa235e4de0a1fa4bb9be08954e => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.821346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.834770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:55.902314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478739c3-8c57-44ba-b58b-1c55b46e0809 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:55.946880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:55.947536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:55.947811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:55.948081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:55.949277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:55.950716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:55.952366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 177/372] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:55.953207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:55.963944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af93a-0c9e-4208-b4dc-5f665a029463 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.005102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.005682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.005963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.006346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.008252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.009729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.011636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 196/373] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:56.012427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.022646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.067919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69e3b-2ebb-4c96-a080-8d4527bf820b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.082634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.106742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.107360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.107639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.107957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.128519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b566ec-5ac3-489c-beb4-3c56e56f5f17 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.168633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.169285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.169577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.169863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.170749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=e38b7fd07d644392bc554b82457fcc43) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:56.171064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.172627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.179849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 197/374] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:56.180667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.186166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.187752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.263605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.298686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b926db-e9fa-42ac-b813-26bc0827301c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.299676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 178/375] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:16:56.306079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.322078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.339394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.340057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.340337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.340625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.341827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.343303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.344975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 198/376] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:56.353683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.466730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c064e-da8b-4275-945c-ff64bb2bb34d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.528076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.528770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.529124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.529409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] PATH_INFO: `/v3/projects/70f7d9422ed44219ac78a91471e7dbb9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.530478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9485-e7e8-44fc-a3f8-41bd82db56c8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.530655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] RBAC: Authorizing `identity:get_project(project_id=70f7d9422ed44219ac78a91471e7dbb9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.532115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7eeea-70ea-44ca-9229-1d35f9ed58b9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.534218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 179/377] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/projects/70f7d9422ed44219ac78a91471e7dbb9 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:56.535588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656e34-fcfc-42a5-985a-90311c160d39 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.535873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656e34-fcfc-42a5-985a-90311c160d39 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.536146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656e34-fcfc-42a5-985a-90311c160d39 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.538715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 180/378] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:56.539892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e656e34-fcfc-42a5-985a-90311c160d39 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.587907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.589893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.590387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.590866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.591892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=e38b7fd07d644392bc554b82457fcc43) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:56.592407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.594606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e4c38-d7a8-4e3b-8ca1-b7c8b4e78243 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.604652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 199/379] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=2d3d839c7e5143618bb567643903898d => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:56.609039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.609039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.609039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.624262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.648795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:56.650728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:56.672640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:56.702426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e656e34-fcfc-42a5-985a-90311c160d39 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:56.768893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:56.769624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:56.769905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:56.770177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] PATH_INFO: `/v3/groups/554dce9063ca47d387fa8ab0786e764c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:56.781971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] RBAC: Authorizing `identity:get_group(group_id=554dce9063ca47d387fa8ab0786e764c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:56.785660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:56.788789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 181/380] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/groups/554dce9063ca47d387fa8ab0786e764c => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:56.792805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 200/381] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:56.795626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.843841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:56.960680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb297a2-4738-49d4-ad5a-4a691ad806d2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:57.019640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fc9ec-df49-4a36-80a4-f1c35489f9b8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:57.024919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:57.025574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.025850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.026121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.026929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] RBAC: Adding query filter params (group.id=554dce9063ca47d387fa8ab0786e764c, role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=70f7d9422ed44219ac78a91471e7dbb9) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:57.027235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:57.028689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:57.042891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 182/382] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/role_assignments?group.id=554dce9063ca47d387fa8ab0786e764c&scope.project.id=70f7d9422ed44219ac78a91471e7dbb9&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:57.071349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:57.072013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.072291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.072570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.073326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:57.089370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:57.094706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:57.105072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:57.105903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 201/383] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:57.106570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:57.125382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:57.259366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f2a67-4419-49b1-8c09-f4ebbb806270 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:57.296935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:57.297522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.297801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.298076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.298802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:57.299104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:57.300469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:57.313803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 183/384] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:57.319154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:57.473048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f41f-d773-4825-9b5a-1d1e034e83e9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:57.524341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:57.525008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.525292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.525564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.526747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:57.528194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f2845-d253-4edb-823a-e876e2bacfca None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:57.531078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 202/385] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:57 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:57.535647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:57.668262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:57.700902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb1d0-bcd1-44ad-a078-ad2ae3477566 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:57.725820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4e8bd-881a-41cc-a337-7551590852a6 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.726749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4e8bd-881a-41cc-a337-7551590852a6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.727035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4e8bd-881a-41cc-a337-7551590852a6 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.733099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 203/386] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:57.742220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.742502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.742772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.750868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:57.751512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:57.751790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:57.752062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:57.754587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:57.756021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:57.759702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 184/387] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:57 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:57.766170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:57.779502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:57.780698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:57.879058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 204/388] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:57.879678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:57.914488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:57.941482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d58d5-9209-460d-96ee-4b2433e43d9b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.011575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.012259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.012539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.012814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.016767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, group_id=049b2e81c34d488ba6f6118c35a0f7a6, role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:58.019011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:58.065451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef48dc3-81b2-44e8-807c-a84cfeffb0bf None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.103050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.103751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.104034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.104306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.107707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:58.113633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 185/389] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 14:16:57 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/2d3d839c7e5143618bb567643903898d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:58.126813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:58.133063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:58.134369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:16:58.135233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 205/390] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:58.142298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:58.297182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa4e89-23dd-4f17-b488-c8d49c8733a1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.334616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.335365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.335694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.335977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.336705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:58.337031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:58.338412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:58.347280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 186/391] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:58.353024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:58.414483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:58.473606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ccc70-1e2f-4723-ab7b-826949a2b796 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.524159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.524857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.525169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.525451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.526623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:58.528070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:58.530043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 206/392] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:58 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:58.535382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:58.632822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:58.688651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5b991-8895-42dc-a41e-01be8a130e61 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.736899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.737567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.737846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.738122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.741758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:58.743196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d5f3a-c971-4875-ada9-7ee29c5fcb7c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:58.745243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 187/393] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:58 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:58.750752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:58.869554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c08d-a98d-49e9-8ed9-9e91daa01ef3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:58.875808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a3ca-f9b8-4f28-b969-34e7b77816bb None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.876088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a3ca-f9b8-4f28-b969-34e7b77816bb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.876361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a3ca-f9b8-4f28-b969-34e7b77816bb None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.879103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 188/394] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:58.883251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.883423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.883700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.888171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:58.901920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:58.903226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:58.908875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:58.914247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:58.914247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:58.914247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:58.914247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=719a4fb20cae4a80bbb5f3922c345311, group_id=049b2e81c34d488ba6f6118c35a0f7a6, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:58.916031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:58.985104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 189/395] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:16:59.004291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.056386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 207/396] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 14:16:58 2022] PUT /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:16:59.075441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:59.121681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34028cf0-f5d3-4f33-a2d0-dd1281553fdb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.159390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.160071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.160351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.160628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.174040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1eb927d2-1986-4e27-be18-9eafee29f40f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:59.179177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:59.180127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:16:59.180977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:59.185096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 190/397] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:16:59.187000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.269449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c7f3-4f4f-4008-b01c-2cf9699c3770 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.269755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c7f3-4f4f-4008-b01c-2cf9699c3770 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.269990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c7f3-4f4f-4008-b01c-2cf9699c3770 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.271887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 191/398] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:16:59.280760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.281117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.281344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.314477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:59.315522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19daf-c393-4d3e-b6ad-1967a57cf7f3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.315716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:59.360450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.361460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.361974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.362448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.363858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:59.364371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:59.368149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:59.379139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 208/399] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:59.391663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.413287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 192/400] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:59.435022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.470956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:59.527490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:59.605312 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2cc5-72e5-4bcb-be65-1712f21f6d8f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.605533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe15cb-9152-4415-81e9-38f384b4f0f4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.644873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.645645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.645927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.646205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.657166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.657807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.658084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.658355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.659744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:59.661207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7a83a-521d-4462-adc7-b8c6a915fa9a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:59.661789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:16:59.662793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 209/401] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-5c631740-771d-4ece-a2b1-cec7eb36d290 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:16:59.668181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:16:59.669346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:16:59.670154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:16:59.670780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95727-188f-40a7-908a-70f439a30f80 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.670780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95727-188f-40a7-908a-70f439a30f80 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.670780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e95727-188f-40a7-908a-70f439a30f80 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.675168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 210/402] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:16:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:16:59.675168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e95727-188f-40a7-908a-70f439a30f80 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.677087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 193/403] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:16:59.678549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.679639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.680166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.700550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:16:59.702237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:16:59.777312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 194/404] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:16:59.778537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.805296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e95727-188f-40a7-908a-70f439a30f80 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.817740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:59.829323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:16:59.846164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.846946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.847292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.847670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.848885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:59.850473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:59.857210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 211/405] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:16:59.858108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.902321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636cc89-305f-4b56-aa2e-34cefc586c9f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:16:59.947092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:16:59.947730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:16:59.948107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:16:59.948442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:16:59.949354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:16:59.949742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:16:59.951176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:16:59.960635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 195/406] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:16:59.961560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:16:59.987811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaff75-a327-4c57-ab8d-01222aefde21 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.037400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.038058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.038352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.038630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.056756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:00.062460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:00.063339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:00.064144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:00.068437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 212/407] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:16:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:00.069289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.071224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.076180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84f3dc-1c58-4497-a41d-d24d4725ac8c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.110912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.123496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.124967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.125446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.125997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.128468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, group_id=049b2e81c34d488ba6f6118c35a0f7a6, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.130667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.182697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc047d-9b3d-4f8a-985a-2469aca41843 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.224601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 196/408] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 14:16:59 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:00.225466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.226046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.226757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.227054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.227310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.228929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.232520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.232520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 213/409] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:00.242174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.363639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248831bb-28a9-43b3-8cc6-c48c6f9865a9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.410156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.410967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.411349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.411729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.413077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:00.414019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.415983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.433281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 197/410] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:00.433746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.448890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff152bd-e44e-48bd-8082-cf87e231171c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.452318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.465619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.495407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.495985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.496266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.496552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.497992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.500273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.505184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 214/411] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:00.510285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.585725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d65c12-cead-465a-8fc7-1af2a43d0ae6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.643064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.644066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.644559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.645120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] PATH_INFO: `/v3/groups/554dce9063ca47d387fa8ab0786e764c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.646534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] RBAC: Authorizing `identity:get_group(group_id=554dce9063ca47d387fa8ab0786e764c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.648796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.655141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 198/412] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/groups/554dce9063ca47d387fa8ab0786e764c => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:00.656542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.698491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b01f-0813-4dd8-b101-b1f063e4c8c6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.738597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.739612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.740087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.740573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.743517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e38b7fd07d644392bc554b82457fcc43, group_id=049b2e81c34d488ba6f6118c35a0f7a6, role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.745362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ad84-9b8e-47b7-b6b6-3493690cd0d2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.765551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.775830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c902-02b6-4acc-8ed6-a6d2ec25fdf6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:00.812099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:00.813425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.813943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.814386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] PATH_INFO: `/v3/projects/70f7d9422ed44219ac78a91471e7dbb9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.815710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] RBAC: Authorizing `identity:get_project(project_id=70f7d9422ed44219ac78a91471e7dbb9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:00.817367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e76a14-40f5-4e81-a4e2-de33ffdd1c47 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:00.824668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 199/413] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/projects/70f7d9422ed44219ac78a91471e7dbb9 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:00.824826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedad00c-d48c-4969-86dc-30738cd79644 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.825063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedad00c-d48c-4969-86dc-30738cd79644 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.825580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedad00c-d48c-4969-86dc-30738cd79644 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.826971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 200/414] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:00.827518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedad00c-d48c-4969-86dc-30738cd79644 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.838816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 215/415] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 14:17:00 2022] PUT /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43/groups/049b2e81c34d488ba6f6118c35a0f7a6/roles/2d3d839c7e5143618bb567643903898d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:00.840499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:00.841018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:00.841495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:00.862439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:00.864064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:00.898179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:00.950687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 216/416] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:00.995007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:00.997047 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedad00c-d48c-4969-86dc-30738cd79644 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:01.063116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:01.064208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.064720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.065243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] PATH_INFO: `/v3/projects/70f7d9422ed44219ac78a91471e7dbb9/groups/554dce9063ca47d387fa8ab0786e764c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.067439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] RBAC: Authorizing `identity:create_grant(project_id=70f7d9422ed44219ac78a91471e7dbb9, group_id=554dce9063ca47d387fa8ab0786e764c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:01.069207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:01.155135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 201/417] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Jul 29 14:17:00 2022] PUT /identity/v3/projects/70f7d9422ed44219ac78a91471e7dbb9/groups/554dce9063ca47d387fa8ab0786e764c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:01.157927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23573614-d85b-4f34-a0e5-7fde810c3a31 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:01.211769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:01.212497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.212779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.213077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.231017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:01.241034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:01.242255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:01.243140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:01.247058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 217/418] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:01.257724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:01.305418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:01.423685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14bf6a-91ff-4f6e-ad95-7e6bed65611d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:01.463565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:01.464192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.464531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.464811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.465567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:01.465866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:01.467296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2a6e-bd3a-4ace-b221-bda5f5314d12 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:01.478111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 202/419] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:01.483729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:01.516806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a5e3-881a-4c99-8e06-e298ddb100ef None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.517133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a5e3-881a-4c99-8e06-e298ddb100ef None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.517412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500a5e3-881a-4c99-8e06-e298ddb100ef None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.519004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 203/420] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:01.524795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.525095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.525368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.525781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:01.546800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:01.547984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:01.672803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1548-f23b-43bb-9db0-84c8098e5cad None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:01.689346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 204/421] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:01.729087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:01.732088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:01.732731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:01.737281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:01.737563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:01.742509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:01.886456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:01.887936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:01.918223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:01.955044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76187d66-4261-4a0a-9b1d-247847a9ce0d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.005839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 218/422] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:02.011453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.041987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.042650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.042927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.043202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.071945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:02.076953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:02.077850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:02.078652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 205/423] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:02.086814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.121894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a558e1-ef0a-4a81-bffe-ce8258dd4615 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.147276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.170601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.170601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.170601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.170601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.172747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.174290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.175952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 219/424] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:02.182391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.248482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.290373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee33f0-3740-4e3a-a5bc-25090bd4e50d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.362207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.362837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.363115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.363390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.364122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] RBAC: Adding query filter params (name=member) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:02.364425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.371054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.390340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 206/425] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:02.395891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.407573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5ac2-d8dc-4d6c-bcae-1d5cc7d78c10 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.461739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.462746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.463249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.463723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.465161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.466843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8ef00-cca4-471e-86c4-afc2f886b553 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.468946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 220/426] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:02.470320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcbdc-6180-4444-871c-55b0221d902a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.470807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcbdc-6180-4444-871c-55b0221d902a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.471322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcbdc-6180-4444-871c-55b0221d902a None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.473039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 221/427] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:02.474185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdcbdc-6180-4444-871c-55b0221d902a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.533675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12497bf-e78d-4f94-a3e4-7f35a42d246b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.540489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.573798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.580415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.581358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.581643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.581917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.588461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdcbdc-6180-4444-871c-55b0221d902a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.627041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.628026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.628561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.629064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.630111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=719a4fb20cae4a80bbb5f3922c345311) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:02.630639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.632862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428be8b6-486b-432f-91d9-2bed501af758 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.639776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 222/428] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=2d3d839c7e5143618bb567643903898d => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:02.641304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.641828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.642319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.662226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:02.663865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:02.666864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.669306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.747861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 223/429] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:02.782329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.805184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 207/430] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:02.811828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.849368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.906489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:02.931082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35b3de-dc39-43c5-90ac-605e4fa85290 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.959570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890f89e-b203-4280-a1de-5914e519fad6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:02.968080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.969060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.969534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.970026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] PATH_INFO: `/v3/projects/719a4fb20cae4a80bbb5f3922c345311` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:02.971404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] RBAC: Authorizing `identity:get_project(project_id=719a4fb20cae4a80bbb5f3922c345311)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:02.973054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:02.975381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 208/431] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/projects/719a4fb20cae4a80bbb5f3922c345311 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:02.981337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:02.998680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:02.999123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:02.999411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:02.999689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.011168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:03.017374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:03.018289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:03.019521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 224/432] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:03.029796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.118419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.122204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569c9d1-826f-423a-9aa5-2e6b115e2f05 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.166601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.167508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.168245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.168759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.170796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:03.173330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:03.176255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 209/433] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:02 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:03.178093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c667-d7de-4e76-a2be-07a601682953 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.182859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.194411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.218137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.218759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.219044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.219321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.220166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:03.220475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:03.222064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:03.232135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 225/434] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:03.238636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.319449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91a96d-e314-4d4a-a007-3eb6b017c73f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.357306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.357963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.358240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.358515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.359314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=719a4fb20cae4a80bbb5f3922c345311) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:03.359623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:03.361249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e42c-12a3-4cad-b0c6-e0fd5c8aaa0b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:03.368602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 210/435] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=719a4fb20cae4a80bbb5f3922c345311&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:03.376022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf9d47-8433-48d6-a554-ac8d0d98be82 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.397609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a5b28-2d06-4f6c-9f16-28b0b345e833 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.397976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a5b28-2d06-4f6c-9f16-28b0b345e833 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.398189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a5b28-2d06-4f6c-9f16-28b0b345e833 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.399978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 211/436] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:03.406961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea98b2-e7bc-4ab8-8ba4-a91dd54f5fb2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.407404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea98b2-e7bc-4ab8-8ba4-a91dd54f5fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.407783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea98b2-e7bc-4ab8-8ba4-a91dd54f5fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.423711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.425411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.425839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.426281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.429788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b5ea98b2-e7bc-4ab8-8ba4-a91dd54f5fb2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:03.431510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b5ea98b2-e7bc-4ab8-8ba4-a91dd54f5fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:03.462899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.490644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.517150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 212/437] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:03.519170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abed3f-0bf0-4452-bb92-076235ff71e3 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.519459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abed3f-0bf0-4452-bb92-076235ff71e3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.519732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abed3f-0bf0-4452-bb92-076235ff71e3 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.521059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 213/438] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:03.527137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.527417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.527693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.529831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:03.531471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:03.548456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:03.549667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:03.629443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 214/439] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:03.631566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.653841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 226/440] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:03.655092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.743430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.748062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:03.752565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb147f09-b13f-49f4-8975-7e08267503fa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.779123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52b12-5c15-46af-b739-841f7456047e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.790938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.791586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.791866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.792136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.804602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:03.809880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:03.810777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:03.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 215/441] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:03.812243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.830558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.831211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.831495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.831773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.843350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:03.848231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:03.849258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 227/442] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:03.852463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:03.938058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e91b5-77be-4d81-bb69-4d14415a269d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.972680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7190a-1b3f-4fe8-a456-daa18a694898 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:03.976910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:03.977567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:03.977841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:03.978111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:03.979264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:03.980697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:03.982355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 216/443] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:03.983112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.005389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.010843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.011457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.011735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.012055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.012782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:04.013114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.014489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.015837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.023485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 228/444] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.024273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.082911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194aef4-765e-4c27-910e-29dde5cc8876 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.116454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.117393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.117673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.117942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.118650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:04.118955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.120332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.122387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c365-2b29-4a6a-8cb3-abf6b3b27d70 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.128130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 217/445] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.128918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.159953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.160707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.160859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.161170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.162338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.164317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.166138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 229/446] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.166908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.214062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.231557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.231696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07a0b8-046f-4344-b6dd-d2c886cda17a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.264478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.265116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.265394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.265664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.268426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7729-e299-4268-b99a-5a557aed608e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.303339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.304499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.304779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.305143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.332260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.333749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.372509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.373998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.413015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.432099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 218/447] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:04.433087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.435572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.474280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 230/448] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:04.475027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.538373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88d3bb-f619-40b8-88ff-7e019e4de84b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.572241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.572854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.573161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.573440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.574219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=680f93d19a99495a81d0cf39e7ac3ce5, scope.project.id=e38b7fd07d644392bc554b82457fcc43) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:04.574531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.575821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82012af6-f306-493d-8f72-f845e3de6b0e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.575974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.581663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 219/449] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=680f93d19a99495a81d0cf39e7ac3ce5 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.582521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.624982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.625759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.626114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.626466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] PATH_INFO: `/v3/projects/e38b7fd07d644392bc554b82457fcc43` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.627801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=e38b7fd07d644392bc554b82457fcc43)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.629407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.631373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 231/450] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/projects/e38b7fd07d644392bc554b82457fcc43 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.637441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.637673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.673129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.702684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875413af-2932-4a8a-8d02-5a4fe41093ad None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.744719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.745402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.746005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.746005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] PATH_INFO: `/v3/projects/70f7d9422ed44219ac78a91471e7dbb9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.747203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] RBAC: Authorizing `identity:get_project(project_id=70f7d9422ed44219ac78a91471e7dbb9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.749029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.750716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 220/451] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/projects/70f7d9422ed44219ac78a91471e7dbb9 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:04.755873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.765571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cbfbf-3b47-4d9d-8efa-cdc0406fd9e6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.802631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.803275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.803559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.803932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] PATH_INFO: `/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.805898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] RBAC: Authorizing `identity:get_group(group_id=049b2e81c34d488ba6f6118c35a0f7a6)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.807352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.813485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 232/452] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/groups/049b2e81c34d488ba6f6118c35a0f7a6 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:04.815138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.879703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ac2b4-8c0b-4db9-8d0c-774fcb64d82c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.894722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.923684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.925402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.925677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.925959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] PATH_INFO: `/v3/groups/554dce9063ca47d387fa8ab0786e764c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.927163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] RBAC: Authorizing `identity:get_group(group_id=554dce9063ca47d387fa8ab0786e764c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.928621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.930499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 221/453] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/groups/554dce9063ca47d387fa8ab0786e764c => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:04.936271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:04.940742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:04.948750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bbc7f-450e-4935-abd4-f8930aa62016 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:04.983946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:04.984554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:04.984836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:04.985177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:04.985953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] RBAC: Adding query filter params (group.id=049b2e81c34d488ba6f6118c35a0f7a6, role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=e38b7fd07d644392bc554b82457fcc43) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:04.986257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:04.987704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880166c-8f19-44ba-886c-a2dd7192ba19 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:04.993759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 233/454] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/role_assignments?group.id=049b2e81c34d488ba6f6118c35a0f7a6&scope.project.id=e38b7fd07d644392bc554b82457fcc43&role.id=2d3d839c7e5143618bb567643903898d => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:05.045991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fd5d9-196a-423a-9eaa-57afa05f7b0b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:05.082479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:05.083109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:05.083389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:05.083663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:05.084462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] RBAC: Adding query filter params (group.id=554dce9063ca47d387fa8ab0786e764c, role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=70f7d9422ed44219ac78a91471e7dbb9) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:05.084771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:05.086804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd138f-7029-4d71-b78d-506b1cabc3e3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:05.093292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 222/455] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Jul 29 14:17:04 2022] GET /identity/v3/role_assignments?group.id=554dce9063ca47d387fa8ab0786e764c&scope.project.id=70f7d9422ed44219ac78a91471e7dbb9&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:07.637197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfd4b-dd54-47c2-8190-6e4f9e503fb7 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:07.638222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfd4b-dd54-47c2-8190-6e4f9e503fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:07.638724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfd4b-dd54-47c2-8190-6e4f9e503fb7 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:07.640576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 234/456] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:07.645501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf14af-25e5-48cb-94ba-fb600133924d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:07.646308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf14af-25e5-48cb-94ba-fb600133924d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:07.646792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf14af-25e5-48cb-94ba-fb600133924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:07.666770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7acf14af-25e5-48cb-94ba-fb600133924d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:07.668291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7acf14af-25e5-48cb-94ba-fb600133924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:07.713925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:07.743663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 223/457] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:07.781739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfd4b-dd54-47c2-8190-6e4f9e503fb7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:07.914675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918e7f-565d-4516-babc-23ded889720f None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:07.914989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918e7f-565d-4516-babc-23ded889720f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:07.915152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97918e7f-565d-4516-babc-23ded889720f None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:07.916582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfd4b-dd54-47c2-8190-6e4f9e503fb7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:07.916864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 224/458] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:07.922149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7869f-2579-40ed-bf76-2f08aaed7726 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:07.922645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7869f-2579-40ed-bf76-2f08aaed7726 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:07.922927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7869f-2579-40ed-bf76-2f08aaed7726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:07.960245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c7e7869f-2579-40ed-bf76-2f08aaed7726 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:07.963048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:07.963960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:07.964435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:07.964904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:07.966484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c7e7869f-2579-40ed-bf76-2f08aaed7726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:07.987992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:07.991262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:08.018996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:08.019930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:08.020773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:08.020773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:08.020773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.020773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:08.023691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 235/459] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:08.031670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.056510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.115855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 225/460] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:08.144133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb5ac-70e6-4fcb-ab27-247df0473f1e None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.144397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ddcae-8811-441a-8293-a96e2639f986 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.144825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb5ac-70e6-4fcb-ab27-247df0473f1e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.145344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb5ac-70e6-4fcb-ab27-247df0473f1e None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.146900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 226/461] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:08.156692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.156901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.157145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.178074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:08.179595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:08.182870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.183700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.184412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.184857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.185791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:08.186300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.188114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.201744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 236/462] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:08.202931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.257736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 227/463] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:08.258995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.287321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.297296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.313632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79707-68e9-42e0-af4d-9d2ad6cc1204 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.350508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.352049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.352597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.353124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.364108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.365891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.368362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee404e-4820-4898-b233-6c5cf7306392 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:08.382868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:08.383833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:08.384698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:08.384698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:08.384698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.384698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:08.386946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 237/464] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:08.388072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.404886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.405558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.405841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.406121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.406791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.408224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67fde7-8b15-4327-b55c-78a94971528e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.493953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c8152-a989-483e-9414-5d7b909e374a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.506669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 228/465] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:08 2022] POST /identity/v3/users => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:08.515452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6ccbe-5ca9-4be5-b1d0-26025b4975a2 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.515925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6ccbe-5ca9-4be5-b1d0-26025b4975a2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.516205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6ccbe-5ca9-4be5-b1d0-26025b4975a2 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.517798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 229/466] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:08.518419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6ccbe-5ca9-4be5-b1d0-26025b4975a2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.523760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.536949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.538257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.539299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.539781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.552948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.555980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-5aee14fe-ebd6-42ae-ab26-4b4564db04c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:08.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:08.574667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:08.575529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:08.575529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:08.575529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.575529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:08.578300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 238/467] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:08.580046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.580564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.581095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.603723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:08.605374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:08.661384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.673988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6ccbe-5ca9-4be5-b1d0-26025b4975a2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.694146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 239/468] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:08.697902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.718042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.718681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.718959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.719239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.720039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:08.720360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.721838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.740465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 230/469] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:08.740465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.818800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9a52-9e3d-423e-b3da-be6855d6d2de None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.830396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:08.851776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafddc0-a694-4bca-8d02-1e68a5701e2d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:08.858685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.859824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.860488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.861507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.862795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:08.863548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.864926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.878148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 240/470] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:08.879374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.888663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:08.889346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.889632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.889908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.902576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:08.904032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-4c92e1c6-d7bd-49e8-a814-60dd90c085d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:08.919869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:08.932011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 231/471] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477fa4-73e6-47b5-b50d-67b5d8093b87 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477fa4-73e6-47b5-b50d-67b5d8093b87 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477fa4-73e6-47b5-b50d-67b5d8093b87 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:08.932885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 232/472] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:08.933422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a477fa4-73e6-47b5-b50d-67b5d8093b87 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:08.933422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:09.002884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e055b9-6da5-4186-bbc0-42b3e85a0d11 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.035004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a477fa4-73e6-47b5-b50d-67b5d8093b87 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.037158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.037770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.038052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.038421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.039257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:09.040701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:09.074627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.074627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.074627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.074627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.075060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:09.075060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c57df-752d-4983-be27-3ed9677fdff6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:09.144418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 241/473] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:08 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:09.145392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:09.177534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 233/474] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:08 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:09.179056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.179431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.179751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.190254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:09.217614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:09.218865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:09.307015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bac3f-d5ce-466b-9098-675bc055d73a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.353203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:09.373791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 234/475] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:09.388539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.389250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.389530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.389807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.390571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:09.390882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:09.392586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:09.397337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:09.407638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 242/476] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:09.414019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:09.526473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:09.528301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd22e6a-2515-446e-95a8-2aa25f536762 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.543844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e3b-dcde-4922-bd2d-e6c72e007c6a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.573617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.574595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.575155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.575436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.576337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:09.578321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:09.585805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.586523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.586908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.587239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.588082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:09.589803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:09.615192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.sql.core [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 14:17:09.615192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 14:17:09.615192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [parameters: {'id': 'ed0a69b876cb4c06a7c348ccb6b412a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 14:17:09.615192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80330) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application result.read() Jul 29 14:17:09.631917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return ref.to_dict() Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:17:09.633645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application yield resource Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise self.value Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application session.commit() Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 14:17:09.634599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self.session.flush() Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application compat.raise_( Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 14:17:09.635473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:17:09.636358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application result.read() Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:17:09.637244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [parameters: {'id': 'ed0a69b876cb4c06a7c348ccb6b412a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:09.638025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 29 14:17:09.639143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.640031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 235/477] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 14:17:09 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:09.643019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:09.684678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 243/478] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:09 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:09.713461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:09.765316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625694-cafa-4b12-bc40-f69b541b2dca None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.808205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.809480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.809812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:09.810137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:09.822565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-3dfaf423-7b21-493c-8a98-e5b96bdded3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:09.827870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:09.828863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 29 14:17:09.829687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:09.830300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 236/479] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:09.836080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:09.961403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155519b-919b-482d-9c4f-a179a40a5c82 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:09.998040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:09.999016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:09.999503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:10.000003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:10.000980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:10.001515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:10.003084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6622d8d-7e34-47be-9857-9f4d728cb15c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:10.012452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 244/480] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:10.837211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565d3d-6450-4784-b6cb-97f6d169013a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:10.837729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565d3d-6450-4784-b6cb-97f6d169013a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:10.837729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565d3d-6450-4784-b6cb-97f6d169013a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:10.839098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 237/481] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:10.848102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:10.848102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:10.848102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:10.863841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:10.885791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:10.887422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:11.006675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 245/482] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:11.037560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df565d3d-6450-4784-b6cb-97f6d169013a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:11.245736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df565d3d-6450-4784-b6cb-97f6d169013a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:11.269396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:11.302598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:11.303249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:11.303525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:11.303794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:11.319056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:11.324738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:11.325667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:11.326495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 238/483] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:11.333552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:11.482352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ccb28-f634-4f45-b896-df1902926ab6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:11.534253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:11.534885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:11.535170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:11.535444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:11.536175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:11.536483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:11.537885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:11.540096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:11.552711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 246/484] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:11.558849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:11.697007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3a8d-880b-4505-9f24-aa32893cf1e1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:11.733340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:11.733901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:11.734181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:11.734458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:11.745127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:11.746912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b3a2c5ef-d0cd-488c-9101-289b26f2a9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:11.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:11.763320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:11.764166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:11.764166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:11.764166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:11.764166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:11.764166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 239/485] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:11.774156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:11.810478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589daa-89f2-44de-b0da-99d12bc4b316 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:11.810788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589daa-89f2-44de-b0da-99d12bc4b316 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:11.811035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589daa-89f2-44de-b0da-99d12bc4b316 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:11.813046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 240/486] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:11.819749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:11.820034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:11.820326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:11.848372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:11.850319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:11.873559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:11.929382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 241/487] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:11.929818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559ab6-84bf-4371-9f95-90ff49908711 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:11.947146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.003151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.003690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.003964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.004240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.004940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:12.005275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.006814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.033475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 247/488] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:12.039178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.058043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546566-cd34-4e1a-8a32-097ae11989d5 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.095236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.096260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.096741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.097249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.109107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-2b0350a1-e778-46d8-8242-6607157b4400 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:12.114482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:12.115393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:12.116207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:12.123440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 242/489] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:12.123440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed72c93-1a95-4f4e-9db9-9efb4c62d408 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.123440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed72c93-1a95-4f4e-9db9-9efb4c62d408 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.123440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed72c93-1a95-4f4e-9db9-9efb4c62d408 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.123440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 243/490] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:12.125370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed72c93-1a95-4f4e-9db9-9efb4c62d408 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.149396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.150428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.186354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed2ee0-0be1-4ddf-8588-3c4aff9ba3ef None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.224144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.224794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.225101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.225375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.233746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.235428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.242469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed72c93-1a95-4f4e-9db9-9efb4c62d408 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-ea8db31c-37ea-4942-87c3-8c432cdbe9bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:12.249046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:12.250001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:12.251516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 248/491] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:12.252096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.252096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.252429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.273607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:12.274870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:12.284601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.285600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.286084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.286555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.287479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:12.287983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.289758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.303199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 244/492] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:12.304563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.350383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 249/493] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:12.351797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.389669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.404251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.412555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85a64f-8659-479d-abfc-7e90ceb010f3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.452344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.453429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.453810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.454158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.455403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:12.456058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.457907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.465280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef807667-0824-41eb-b709-2c0b3e5a7dec None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.504560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 245/494] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:12.505777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.514461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.515117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.515399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.515679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.529840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.531792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:12.547810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:12.548699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:12.549571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:12.549571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:12.549571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:12.549571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:12.551427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 250/495] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:12.552548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.618411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364151f9-f8bf-4021-a832-e182b4fad2ea None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.658763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.659309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.659884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.660180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.660611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.663929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.665331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9fcfe-cdae-4b69-b1ae-c70ae7444f63 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.671953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-15cc20a6-d02f-44df-9aea-425c56d40f40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:12.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:12.678028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:12.679545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 246/496] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:12.680413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d8d9-141c-46bf-8de1-6823a42fdf63 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.680679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d8d9-141c-46bf-8de1-6823a42fdf63 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.680966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9d8d9-141c-46bf-8de1-6823a42fdf63 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.682405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 247/497] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:12.682925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9d8d9-141c-46bf-8de1-6823a42fdf63 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.710540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.711182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.711461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.711806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.787117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.788541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17adc0-46ee-4ac7-a162-3aa2a4eeb332 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.789864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9d8d9-141c-46bf-8de1-6823a42fdf63 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.825333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.825942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.826219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.826492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.827185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:12.827488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.828857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.839911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 248/498] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:12.839911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.895918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.898927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:12.903497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 251/499] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:12.904777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.905089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.905472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.923733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:12.924924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:12.942067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1525ff2-dd33-4fb3-91b0-037d353e96d0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:12.974407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:12.975016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.975290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.975575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.976983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:12.977321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:12.978676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6465c-9a59-4c75-8851-d9ce64518871 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:12.987180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 249/500] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:12.988372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070044-4221-43b8-8532-a4055052ad9f None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:12.988712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070044-4221-43b8-8532-a4055052ad9f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:12.989014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070044-4221-43b8-8532-a4055052ad9f None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:12.990237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 250/501] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:12.990936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070044-4221-43b8-8532-a4055052ad9f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:12.998672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 252/502] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:12.999514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.095981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070044-4221-43b8-8532-a4055052ad9f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.101989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d921-b6e7-4e39-92e4-bbd2809215cd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.112466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.128681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.128934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.129585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.129863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.130138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.136622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.137685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.137685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.137870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.138933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.140421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.147610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.149804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:13.153687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.154571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 251/503] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.156630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-9eb4c530-e2a1-4a31-a401-5ff152be1df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:13.170939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.172033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 253/504] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.173592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.173908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.173971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.191746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:13.192923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:13.259465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 254/505] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:13.260049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98a6c-6065-4575-bb1b-e51b1f199e0c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.260337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.294300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.294962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.295274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.295559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.306495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.307945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:13.322695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.323579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.324435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:13.324435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:13.324435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.324435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:13.324435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 252/506] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.324716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.354769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.361603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a394-86b5-48ee-acd5-f936bc3581bf None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.372606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.395229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.395876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.396159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.396433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.407107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:13.412040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.412982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:13.413999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:13.414652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 255/507] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.415014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.428013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cbf23-c3a6-4a92-9fd8-23750b518c45 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.463053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.463662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.463941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.464216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.465030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:13.465341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.466770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.483290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 253/508] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:13.484090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.515651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8de76-a887-4793-95f4-e20b704b7c98 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.549425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.550033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.550312 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.550585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.551296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:13.551598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.552969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.566822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 256/509] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:13.567622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.575969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.588256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82c5f1-f520-4079-85e0-ff825eb92196 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.597984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.624099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.624677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.624960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.625280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.635780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.637263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:13.652242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.653241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.654092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:13.654092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:13.654092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:13.654092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:13.654092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 254/510] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.654373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.674816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcabbf1-eae8-4fd8-99e3-ba5a2599c50b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.709218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.709828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.710111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.711216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.711903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:13.712205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.713614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.724598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 257/511] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:13.725417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.755348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd79f1-b7fd-41f0-bf25-5bb2dcea2e59 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.787658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.788802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.789310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.789639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.790348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:13.790810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.792183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.799894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 255/512] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:13.800673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:13.806788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.821288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:13.830538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1ed0c-f715-48ca-b572-a2ec87d38712 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.863829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.864449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.864733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.865034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.905846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4694b-808f-421e-beba-64217ea3db6e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:13.933517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.935454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.939368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:13.940011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:13.940292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:13.940568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:13.948936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:13.950419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:13.963004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:13.963950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:13.964806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 256/513] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:13.965537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.028799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.031004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.040883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 258/514] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:14.041736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.066929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799a81-929a-4194-a6bf-a6bf653f97b1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.102296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.102905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.103180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.103457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.104142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.104448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.105845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.117239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 257/515] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.118030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.141343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9644-fa8d-475d-ab53-4dd4d40fc30e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.177706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.178269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.178548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.178826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.186236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.187671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:14.199556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:14.200531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:14.201464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 259/516] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:14.202044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.223587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58eff6-e97b-48f7-b3af-4ad62eb217d6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.246142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.254614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.257778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.258463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.259929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.259929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.268680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.270156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:14.286160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:14.293412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 258/517] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:14.294342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.304533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec36e-432d-46ca-9eac-c08bec896814 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.338948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.339564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.339837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.340108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.340846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.341181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.342612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.360887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 260/518] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.361707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.396077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7125e3-88d5-46ef-bca6-9fb31f9e7c2f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.429634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.430434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.430710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.430982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.441440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.442881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:14.457261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:14.458140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 259/519] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:14.459288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.463768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fd51b-a107-49e6-851d-37c24b0d47e7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.474648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.487001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.498388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.498986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.499266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.499540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.500208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.501684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.557056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb752cc-8d29-45bb-a5b8-07ee9dc33425 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.583782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 261/520] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:14 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:14.584780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.591498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.592138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.592416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.592685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.593543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.593853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.595327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.616448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 260/521] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.617707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.714768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc5df-b4b7-490f-9ce8-6116a53653a1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.715278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.732780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbeddf-7534-48e8-8aac-fc267d311c98 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.754317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.754978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.755221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.755489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.756183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.756482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.758612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.771434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 262/522] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.772066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.779389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.780015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.780289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.780559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.781407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.781709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.783129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.790035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 261/523] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.790845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.806918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:14.901819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafac4b-1a0d-496c-83a9-3e4bff866554 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.943816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a21f5-c07e-45a3-9b9c-f152a62778ec None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:14.944934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.945780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.946160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.946533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:14.947492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:14.948816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:14.950267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:14.966498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 263/524] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:14.967541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:14.990424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:14.991152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:14.991598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:14.992021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.015701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.065207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.084063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c57b-f02f-473a-ba85-a419493da79c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.086799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.088335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.124145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.125211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.125694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.126188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.134170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.135926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:15.149489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:15.150441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:15.151367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:15.153676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 264/525] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:15.155311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.207456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 262/526] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:15.208428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.258102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.269652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4125f2-cfaa-4221-9791-304c2e17f27f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.306246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.321983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.321983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.321983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.322357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadcdc8-71e5-4794-be61-e68dc433c4c4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.322434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.322434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:15.334491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:15.335950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:15.337053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:15.344077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 265/527] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:15.344077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.354564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.355191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.355478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.355754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.356501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.356607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:15.356852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.358843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.382750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 263/528] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:15.383640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.471473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff873-8736-48f2-88ac-450dd769300a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.510988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156eac4-1276-4146-86f5-8c5c411f17ec None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.515228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.516129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.516634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.517141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.518788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.531504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.533555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.555520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.556136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.556412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.556681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:15.557220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:15.558060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:15.558909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:15.558909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:15.558909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:15.558909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:15.559390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:15.559390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.559390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.561967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 266/529] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:15.563023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.590886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 264/530] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:15.590886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.631914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.677187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44663f42-1793-4fd3-886e-a6c1bec20f35 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.704070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496824-b3de-4ae7-9a20-8dae095d0ce6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.715299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.715705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.715987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.716256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.743626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.745023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.745804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.746285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.747844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:15.748496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.750494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.768471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 265/531] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:15.769635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.796904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.798419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.806624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.867688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:15.887119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4ac5-f4eb-4141-bc3e-57f2f0279328 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:15.918455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 267/532] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:15.919338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:15.928705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:15.929938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:15.930222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:15.930573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:15.931293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=a86fe5edd283422cb3d5650845927693) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:15.931604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:15.933075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:15.945058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 266/533] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/role_assignments?user.id=a86fe5edd283422cb3d5650845927693&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:15.945058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.087012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c66d-09e4-4f71-9672-7a2ca4392709 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.098393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f251a69-e691-462b-a714-5fa215d35773 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.101924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:16.140437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.142624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.142923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.143199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.166738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.166738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.167844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.168671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.169126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.180981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.184897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:16.196458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:16.197406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:16.198440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:16.198440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:16.198440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:16.198440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:16.204302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:16.205343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:16.206423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:16.207067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 267/534] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:16.208225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 268/535] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:16.209341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.214494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.235850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:16.351321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb08f1-fdfb-445f-85ef-889f59216d7f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.390707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.391337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.391619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.391894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.397022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:16.397022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.397022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.408984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 269/536] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:16.411852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d99a-59f2-46f1-b69a-efaa4054d333 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.419077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.502729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.503414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.503699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.503970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.504675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:16.504976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.506383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.509486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:16.529657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 268/537] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:16.536799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.545315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:16.546157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334adf1d-1191-4849-8123-53a729a69e8b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.614297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.614837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.615118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.615582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.616374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=c7c0f85f0b514af0bc2de51aeaf4ffdf) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:16.617252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.618847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079eadf-ff68-4b2e-822f-5773c2c53358 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.631414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 270/538] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/role_assignments?user.id=c7c0f85f0b514af0bc2de51aeaf4ffdf&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:16.717352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fecc0f-0628-4801-a21f-c88c6793865f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.764316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f3c7-e3b6-4e5f-9f30-5eac803f0194 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.764625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f3c7-e3b6-4e5f-9f30-5eac803f0194 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.764881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f3c7-e3b6-4e5f-9f30-5eac803f0194 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.766575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 271/539] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:16.772071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.772357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.772672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.773834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.774504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.774781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.775055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.786389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.787964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:16.792245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:16.793526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:16.806349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:16.807275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:16.808351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 269/540] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:16.819746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.901133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 272/541] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:16.904801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:16.942299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:16.947411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63eae21-9068-4d6e-bd11-9c88679a5df0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:16.988009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:16.988678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:16.988965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:16.989266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:16.990026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:16.990336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:16.992345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.007411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 270/542] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:17.018024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.065858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.102031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2604a9e-2019-437d-9af2-978307f6d170 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.135928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1313079-d8fa-4848-aef2-32abd17dc102 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.138210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.138867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.139144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.139417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.150533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-8fa3c897-4bd5-4367-9645-99dba0f09a02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:17.155701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:17.156564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:17.157363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:17.157967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 273/543] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:17.161103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f8d3-32fa-4f8f-abd0-1b8db37a8cf8 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.161103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f8d3-32fa-4f8f-abd0-1b8db37a8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.161103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103f8d3-32fa-4f8f-abd0-1b8db37a8cf8 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.161547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.173234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 274/544] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:17.173234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103f8d3-32fa-4f8f-abd0-1b8db37a8cf8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.179198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.179809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.180084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.185959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.185959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=06cce26ee600472f84a9d8f7b782bdb0) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:17.185959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:17.185959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd775a33-e83c-4e16-90cb-bda419fa0568 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.189690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 271/545] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/role_assignments?user.id=06cce26ee600472f84a9d8f7b782bdb0&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:17.191095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.191486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.191765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.219409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:17.221068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:17.275394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103f8d3-32fa-4f8f-abd0-1b8db37a8cf8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.310029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.315055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.315685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.315964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.316248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.316971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:17.317349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:17.318791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.328902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 275/546] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:17.334374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.377615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 272/547] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:17.422186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.456051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd288-0bbb-4785-983b-c598f6ac8698 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.515596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.516292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.516579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.516916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.530364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:17.532348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:17.548812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:17.549808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:17.550644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:17.550644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:17.550644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:17.550644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:17.550858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 276/548] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:17.556556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.613540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.667124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8336a66-1ffa-43dc-a951-1e9ed4a4fe86 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.676943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d2f23-5f8d-4aa6-bb65-91ca61bade8f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.709159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.717937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.718122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.718328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.718613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.719312 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:17.719628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:17.721521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.735910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 277/549] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:17.741333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.778866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.779529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.779793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.780063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.807894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:17.813337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:17.814244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:17.815061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 273/550] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:17.825959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.853992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83d07-d8a7-4422-befc-1cca24e35538 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:17.869330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:17.892177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:17.892834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:17.893143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:17.893421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:17.902106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:17.903811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:17.931592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:17.932683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:17.933624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 278/551] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:17.939335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:17.975078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e299c1-eeec-4634-806c-a9f33f64e632 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.016351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.016973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.017277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.017553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.018276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:18.018579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.019962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.034227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 274/552] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:18.035477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.081377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.133255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd76ab-ae23-40e7-9a3d-01ea0c878e7a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.148279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374908-0982-4f66-98f9-b148250b8ee3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.204226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.204766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.205095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.205372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.206170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:18.206480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.209458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.220738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.221411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.221689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.221963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.242355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.243811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:18.266118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:18.267018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:18.267875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:18.267875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:18.267875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:18.267875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:18.274306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 275/553] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:18.281436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.297626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 279/554] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:18.302711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.349346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.437417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.447297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47c29-c0c8-4f73-ace4-6958f9486b84 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.467894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a685314-9164-4789-9fd5-fc7dda84375e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.485487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ec11b-1e48-411a-a750-210944203590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.486106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ec11b-1e48-411a-a750-210944203590 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.486388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ec11b-1e48-411a-a750-210944203590 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.486664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ec11b-1e48-411a-a750-210944203590 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.487356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ec11b-1e48-411a-a750-210944203590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:18.487669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ec11b-1e48-411a-a750-210944203590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.489111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ec11b-1e48-411a-a750-210944203590 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.502937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 276/555] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:18.504393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc98ca-4440-4502-be0b-cdf9202c7edb None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.504838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc98ca-4440-4502-be0b-cdf9202c7edb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.505278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc98ca-4440-4502-be0b-cdf9202c7edb None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.506902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 277/556] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:18.509391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc98ca-4440-4502-be0b-cdf9202c7edb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.520272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.521389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.521875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.522389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.598703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.600522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.614935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc98ca-4440-4502-be0b-cdf9202c7edb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.629154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.650244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.650890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.651168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.651442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.658871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.660328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-2d5bbf18-1979-43b4-a378-b40d7e1d6877 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:18.674173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:18.675635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:18.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:18.680011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 278/557] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:18.681493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.681633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.681888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.700036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:18.701283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:18.704345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.721661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 280/558] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 29 14:17:18.722988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.777356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 279/559] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:18.778243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.850157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b7b4-869b-4c0c-9e31-ac603f662cca None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.877615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.889368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.890321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.890804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.891278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.892288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:18.892795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.894691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.908582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d22bf-0674-460d-9085-4006cc44dcb3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:18.916750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 281/560] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:18.918004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:18.958207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:18.958861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:18.959141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:18.959417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:18.971269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:18.974573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:18.976565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:18.992525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:18.993501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:18.994410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:18.994410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:18.994410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:18.994410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:18.994410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 280/561] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:18.994693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.025961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76953d-4960-4d7a-8086-36eb7430e315 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.062230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.062863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.063143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.063415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.075285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c60e53dd-ff72-4106-9234-36bd21d5f5e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:19.080633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.081946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 282/562] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.083018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ecb6a-4dd5-4172-ab3f-c21627797a2f None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.083945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ecb6a-4dd5-4172-ab3f-c21627797a2f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.083945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ecb6a-4dd5-4172-ab3f-c21627797a2f None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.089219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 283/563] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:19.089219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ecb6a-4dd5-4172-ab3f-c21627797a2f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.107518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06efbd-5798-46a3-be19-79592bb4585d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.137249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.142559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.143205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.143487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.143763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.156626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.159150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a9ab62a0-7cca-4130-9573-83e44b2a1165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:19.176431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.179850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.180760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:19.180760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:19.180760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:19.180760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:19.180760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 281/564] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.181367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.181657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.181935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.202011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:19.203473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:19.216250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ecb6a-4dd5-4172-ab3f-c21627797a2f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.246008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.255591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.256214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.256497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.256770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.257520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:19.257821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.259208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.281014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 284/565] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:19.281014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.281274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 282/566] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:19.281274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.383982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d832c9-ad63-4ba2-9f35-c8ca92a5aeed None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.387429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f76ef-c534-440b-9cf8-cdbf10bc6f04 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.389126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.421271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.421895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.422175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.422446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.423135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:19.423328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.423458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.423944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.424224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.424492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.424809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.425242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:19.425544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.426907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.440145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 285/567] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:19.440145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.440385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 283/568] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:19.440385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.476892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.544061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567db9-cbc2-4f62-a6a1-123b4a974dc6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.546550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e233-ed88-47a7-baac-d228832d8631 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.579257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.579894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.580167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.580435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.582133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.582725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.583005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.583281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.587658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.589117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.593824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:19.599174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.600085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:19.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:19.602577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.603392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:19.604288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:19.604712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 284/569] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.604712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.611099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 286/570] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.611099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.636896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.708884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.716060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf82a73-082c-4c30-95f0-c25fa6f4bb07 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.716483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc2a5f-2fa8-4c4d-bc7a-f5e73f4cd046 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.751063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.751696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.751972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.752222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.752342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.752861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.753176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.753452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.758963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.760405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.764076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.765531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:19.771403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.772278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 287/571] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.773950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:19.781842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:19.782943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 285/572] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:19.789312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.859880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:19.876021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be40c88-05ec-4029-9940-2efbc4bd03a4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.887603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4ed41-4e9a-40cc-a7fb-7d53b4b06885 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:19.909466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.910081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.910358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.910626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.911329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:19.911629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.913016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd845e02-450f-44d3-a9ac-73dcb7b17a3e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.920737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:19.921416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.921696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.926390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 288/573] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:19.926496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.926496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:19.926496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:19.926681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704981-9ff2-4daf-a44a-65dc4b302182 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:19.926681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704981-9ff2-4daf-a44a-65dc4b302182 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:19.926681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704981-9ff2-4daf-a44a-65dc4b302182 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:19.926869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:19.926936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 289/574] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:19.926936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704981-9ff2-4daf-a44a-65dc4b302182 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.946123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 286/575] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:19.946123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:19.967071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.057306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f01c-e1a2-4c6a-b8b3-fcf5d83545e2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.060686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704981-9ff2-4daf-a44a-65dc4b302182 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.090794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.091429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.091706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.091975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.092659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:20.092957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.093478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.094079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.094353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94476d-dbbe-491a-acf5-0c3253f224fc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.094444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.094624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.095359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:20.095667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.097095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.105550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 287/576] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:20.106709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.106992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.107264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.118635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 290/577] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:20.118635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.122996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.130741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:20.131962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:20.193224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.201847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 288/578] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:20.202866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.227051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69c00-8757-49f2-ac1c-6e2e83d4916a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.262055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.262692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.262968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.263237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.273427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.274869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:20.289075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:20.289990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:20.290894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:20.290894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:20.290894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:20.290894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:20.290894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 291/579] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:20.291179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.305940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca230cd-7bf0-4278-964f-8deec0768436 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.340784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.341452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.341733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.342006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.342801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=3189acc363fe42339aedae0a604c8b42) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:20.343106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.344561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.349888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 289/580] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/role_assignments?user.id=3189acc363fe42339aedae0a604c8b42&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:20.350642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.362473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.402529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161e66-7a26-4bc0-b1dd-14bfb8402b80 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.424948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.443443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.444699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.445530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.445830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.455194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.456672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:20.469487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:20.470696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 292/581] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:20.471725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.480686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0109e8-87e4-4582-940f-ecdaeb36bbc6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.522464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.523193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.523603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.523884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.540371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=e4a3cbcc38d741178be408abd4156959, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.541849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.586018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec7fdc-bbf9-4203-af79-cd0d1b600ef0 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.603061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.619699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.620334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.620614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.620887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.632810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 290/582] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:20 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:20.633361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:20.640350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:20.641361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 293/583] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:20.642240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.644011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.771974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c90d81-1db8-48ac-adba-fd4f15e8eea9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.797230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f176e-089e-4498-91da-9468cdd90c7c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.811174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.821753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.822381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.825519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.825519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.825519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:20.825519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.826176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.838575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:20.839482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:20.839946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:20.840365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:20.841261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:20.841743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:20.843291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:20.846619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 294/584] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:20.847308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.857423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 291/585] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:20.858579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:20.893361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:20.986957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba12552-69a4-4d6b-bd59-3bc02ec29bd7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:20.987719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357002e-893a-44cb-9b47-0a7f4fb3950e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.031251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.031787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.032061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.032328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.032621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.033071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.033306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.033408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.033588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.033867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.034727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.046963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 295/586] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.047881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.053266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.055340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:21.075628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.077022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.077906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:21.077906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:21.077906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.077906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:21.077906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 292/587] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.078622 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.114522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.160649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.161930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c212eb3-e4ff-47b8-b3ee-51901b82ad59 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.197184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.197839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.198120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.198397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.198898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca135d-ab87-48de-82dc-5377181e9087 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.206608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.208069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:21.219667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.220619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:21.221584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 296/588] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.222167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.241057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.242550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.242882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.243225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.255064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.256671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:21.273335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.275495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 293/589] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.277166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.332621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc821a6e-eed4-4510-b535-86df48a2a194 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.359494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.368636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.373324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.384967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 297/590] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.385919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.386483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154be21-cdbe-4422-bc21-b911c7b8d231 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.405328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.425164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.426005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.426424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.426878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.427809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.428445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.430719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.451449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 294/591] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.452670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.492173 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb47ae2-7651-4d9b-8cee-2c515702db73 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.528869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.529183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.529466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.529745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.530452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.530759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.532144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.549850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 298/592] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.550897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.566848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1a542-c723-4bf7-ad77-8bb729961522 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.599599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.606350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.607354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.607831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.608300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.616028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.617743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:21.630213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.631151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:21.632054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:21.634620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 295/593] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.635767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.636843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.660505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b779ed0-6eac-4d5b-a390-5b9b499c3628 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.698310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.698956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.699254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.699518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.711920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.714064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:21.729799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.730691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.731567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:21.731567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:21.731567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:21.731567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:21.731567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 299/594] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.731991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.748321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb6b6-099c-474f-bc66-e08431a799ac None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.786314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.786959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.787326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.787496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.796016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.797496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:21.810684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:21.811661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 296/595] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:21.812983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.861729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9bc57-dfd2-4aef-b663-f756a50571fb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.887975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.896778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:21.903752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.904636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.905174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.905629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.906561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.907070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.908720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.921814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 300/596] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.923194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:21.940344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e99f79-5c67-44f3-b24d-c369fd5bbbca None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:21.978259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:21.978784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:21.979060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:21.979332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:21.980028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:21.980328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:21.981721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:21.995760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 297/597] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:21.996584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.035486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fc410-5716-4e46-abb4-3862a4c270e5 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.072491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.073485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.073973 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.074449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.075447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:22.075982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.077692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.100423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 301/598] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 14:17:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:22.102178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.108842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ba466-c962-4a5b-aa5a-d0ae569799d4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.126083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.151366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.152037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.152312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.152582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/a86fe5edd283422cb3d5650845927693/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.160313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.168793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=a86fe5edd283422cb3d5650845927693, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.170296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.210551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91d3d-8ebb-4dda-a312-c1c94e20049d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.248487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.249626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.250105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.250639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.257296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 298/599] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:21 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/a86fe5edd283422cb3d5650845927693/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:22.258552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.267459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.269370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:22.300777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:22.301792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:22.302674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:22.303945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 302/600] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:22.310159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.441828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21acae-89eb-4b50-a08e-ac721bed1d3c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.452197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40267412-f0ff-47a7-a86e-758dcb5fe0aa None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.463266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.491070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.492117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.492235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.492651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.492844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.493161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.493419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.493527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.494284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:22.494705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.496154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e44af-cd4f-4dfd-8053-4a18aa88edbd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.505711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.507561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.509820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 303/601] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:22.511908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975205c-ed61-4f6f-adb1-90c96c8639cd None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.512204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975205c-ed61-4f6f-adb1-90c96c8639cd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.512491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975205c-ed61-4f6f-adb1-90c96c8639cd None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.514087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 304/602] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:22.516691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975205c-ed61-4f6f-adb1-90c96c8639cd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b6fccb2f-fdb7-417e-a4b7-7dca9290c55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:22.530584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:22.532334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:22.534141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:22.534141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:22.534141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:22.534141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:22.534141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 299/603] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:22.534683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.535558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.535801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.564174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:22.566073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:22.590143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.644842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975205c-ed61-4f6f-adb1-90c96c8639cd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.688923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.689637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.689919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.690256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/c7c0f85f0b514af0bc2de51aeaf4ffdf/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.711874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=c7c0f85f0b514af0bc2de51aeaf4ffdf, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.712308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 300/604] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:22.712668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.714374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.811355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 305/605] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:22 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/c7c0f85f0b514af0bc2de51aeaf4ffdf/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:17:22.812256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.845198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.851714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5273f3-5fcd-4e4d-97e0-42d855c18733 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:22.894878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:22.895827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.896309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.896758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.897756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:22.899530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:22.901197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc4891-d08f-4e51-b89c-9c2034b687af None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:22.924384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 301/606] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:22.926127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb047-f1b0-4d71-ad6b-2258cb7b1da3 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:22.926625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb047-f1b0-4d71-ad6b-2258cb7b1da3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:22.927103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb047-f1b0-4d71-ad6b-2258cb7b1da3 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:22.928921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 302/607] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:22.930001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb047-f1b0-4d71-ad6b-2258cb7b1da3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:22.934977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:22.962344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207de600-b268-4b62-a172-c8e7e6e5fce8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.016411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.017361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.018422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.018422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.030085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c95d8fdf-56e9-4aa6-b304-ec0f8e0dac7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:23.034982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:23.035913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:23.036733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 306/608] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:23.038397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.038397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.038397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.059521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:23.060707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:23.064187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb047-f1b0-4d71-ad6b-2258cb7b1da3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.108617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.110396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.110943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.110943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.119783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.121587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:23.136712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:23.137795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:23.138668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:23.139099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 303/609] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:23.140600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.146041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 307/610] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:23.147016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.172803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.227694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.279208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5f557-9466-4832-bbb5-fe4185bc29fd None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.291059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2e5d-39ed-483b-92f3-d43ac42650c6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.317658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.318279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.318557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.318828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.319606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:23.320627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.322324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.329095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.329728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.330009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.330288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.331044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:23.331216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 304/611] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:23.332006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.335887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.337457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.352377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 308/612] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:23.353391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.425263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.441778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07feb599-6d00-4f73-9b39-69c866ee278e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.463457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bd9c3-51a0-4898-b44d-7588c7a65d1c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.471159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.482060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.482924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.483345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.483773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.496784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:23.499929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.500578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.500860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.501166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:23.503069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:23.504152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:23.505248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:23.507650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 305/613] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:23.508656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.513858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.515330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:23.531340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:23.532436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:23.533532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:23.533532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:23.533532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:23.533532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:23.537187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 309/614] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:23.538976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.620297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8d980-b7c9-4b69-b7fa-509ff06e8e54 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.654022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154cc6f-fb20-4a6d-9836-672d71131bbd None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.658729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.659595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.660016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.660427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/06cce26ee600472f84a9d8f7b782bdb0/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.675259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.678454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=06cce26ee600472f84a9d8f7b782bdb0, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.680818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.690604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.692195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.692195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.692195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.692520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:23.692772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.694181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.703037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 310/615] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:23.703764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.777286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 306/616] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:23 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/06cce26ee600472f84a9d8f7b782bdb0/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:23.783984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.845856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266b08-76b6-45b7-8014-a06f62dbe922 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:23.853373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:23.919720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:23.920358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:23.920638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:23.920916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:23.921713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:23.922083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:23.924443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:23.953135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 311/617] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:23.960823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:23.987035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c916-4e16-4c94-9133-63530e7ca951 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.022009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.042983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.044565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.044841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.045139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.057551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.060402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:24.078160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:24.079310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:24.080585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:24.080585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:24.080585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:24.080585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:24.080585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 307/618] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:24.086680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.098218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c128c8-d2e0-4276-88be-d0984ffd2075 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.136557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.137554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.137984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.138407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.148578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.150381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:24.163149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:24.164915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:24.165935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:24.172471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 312/619] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:24.173891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.275933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.283195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d978c-d1e8-4beb-866c-e1ee8cadf283 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.305179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.305363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621522-6acf-418f-8fbd-d15e7924d047 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.321731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.322600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.323712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.324133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.325126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:24.325648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.327423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.352910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 313/620] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:24.354936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.421189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.421832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.422123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.422401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.423108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:24.423413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.424790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.451704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 308/621] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:24.456610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.465175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2dfeb-fe7c-436b-8ff0-127f45a1dd47 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.503443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.504490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.505041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.505598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.517250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.519018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:24.535582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:24.536543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:24.537496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:24.537496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:24.537496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:24.537496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:24.544917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 314/622] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:24.546524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.565524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.642347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e0b3-89a2-403c-9cca-93a0dfd461fc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.658357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12c8a7-5b13-4ef5-ae3d-0b1194943056 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.696076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.697181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.697703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.698180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.699077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:24.699582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.701216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5f3c8-40e8-4ff8-b75e-68b444134999 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.712760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.713840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.714322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.714814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.727434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 315/623] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:24.728657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.730151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.730652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f8d9-5245-4304-a153-d1bb0f2174d3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.731140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f8d9-5245-4304-a153-d1bb0f2174d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.731613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f8d9-5245-4304-a153-d1bb0f2174d3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.734153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 316/624] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:24.738559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f8d9-5245-4304-a153-d1bb0f2174d3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-9aff8841-7c45-47b2-9292-fad6fd6ad691 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:24.745186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:24.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:24.747115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 309/625] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:24.748170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.748170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.748352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.771503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:24.773573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:24.776414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.829425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:24.861217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 310/626] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:24.867723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.879174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f8d9-5245-4304-a153-d1bb0f2174d3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:24.927490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:24.929742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:24.930239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:24.930956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:24.940147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:24.942762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:24.959419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:24.960384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:24.961579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:24.964159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 317/627] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:24.965567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:24.993757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678cb7-d7c2-4c2e-af04-93c34456e5b7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.044433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189c901-a263-428d-a174-d5617d142669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.045085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189c901-a263-428d-a174-d5617d142669 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.045363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189c901-a263-428d-a174-d5617d142669 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.045643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189c901-a263-428d-a174-d5617d142669 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.046423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189c901-a263-428d-a174-d5617d142669 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:25.046727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189c901-a263-428d-a174-d5617d142669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.048137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189c901-a263-428d-a174-d5617d142669 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.056894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.066684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 311/628] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:25.067479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189c901-a263-428d-a174-d5617d142669 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.091954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d4ed1-e7a6-4da8-852a-f1c4a6fb47f7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.123443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.128883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.129639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.132733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.132733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.142057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-92780131-59bc-4d76-8e84-ce3034974868 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:25.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:25.148724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:25.150370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 318/629] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:25.154094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2209a-f472-4a77-9316-f67591025a7e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.154383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2209a-f472-4a77-9316-f67591025a7e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.154669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2209a-f472-4a77-9316-f67591025a7e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.160833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 319/630] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:25.161615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2209a-f472-4a77-9316-f67591025a7e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.187162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189c901-a263-428d-a174-d5617d142669 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.227282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.227926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.228202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.228483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.229292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:25.229603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.231076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.244675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 312/631] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:25.245567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.275115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2209a-f472-4a77-9316-f67591025a7e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.307783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.312871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.313746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.314035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.314645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.354613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c911-953c-4d7f-b4cb-4350f35cc6e9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.382898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.390854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.395502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.395502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.395502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.395502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:25.395819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.395903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.395980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.396049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32193ee4-b149-404a-98ff-d0c90154fd9d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.408037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 313/632] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:25.408037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.408037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.408037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.424984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:25.426259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:25.497293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 314/633] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:25.502491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.513512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 320/634] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:25.514411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.561206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.612594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.615208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194caf4-9b6e-4be1-bba4-2e9e762a1175 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.625541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210679a4-1a5c-4356-9c28-bbaf94d71850 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.652968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.653778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.654121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.654467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/3189acc363fe42339aedae0a604c8b42/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.662036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.662686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.662967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.663249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.670513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=3189acc363fe42339aedae0a604c8b42, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.672449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.673561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.674996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:25.690889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:25.696545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 321/635] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:25.697719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.757455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 315/636] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:25 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/3189acc363fe42339aedae0a604c8b42/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:25.758688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.812327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16b720-0980-4fe8-8b5e-b47154e6c53a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.852390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.853589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.854025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.854531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.869656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:25.871077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:25.881365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:25.896970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:25.899026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:25.900102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:25.900102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:25.900102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:25.900102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:25.907756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:25.908652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 322/637] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:25.910267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:25.912976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0952c9-653a-4c19-b44f-45b8933a209c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:25.957953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:25.958763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:25.959189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:25.959564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:25.971216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:25.976518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:25.981499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 316/638] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:25.982334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.043129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45612e7e-e2bb-4501-855e-6929ad440b06 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.093994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.094792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.095132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.095419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.096394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.096736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.098901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.103681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24996c2-152f-4bb8-ba08-9cc49d9df524 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.119871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 323/639] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.119871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.154940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.155576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.155857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.158644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.159374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.159778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.161123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.174655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 317/640] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.175481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.196479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.198318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.246661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a60db-cd9e-443d-9645-69d456782b9b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.284491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.285141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.285424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.285529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362b88-4b68-4bd8-95e8-52e1050716d7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.285703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.286424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.286727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.289470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1fb0f-08f5-48a1-ab5d-76d9e1166b7f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.301085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 324/641] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.302448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7881a9-5ae2-4929-b3fb-b99ec52f99d3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.302808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7881a9-5ae2-4929-b3fb-b99ec52f99d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.303166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7881a9-5ae2-4929-b3fb-b99ec52f99d3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.309602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 325/642] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:26.310318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7881a9-5ae2-4929-b3fb-b99ec52f99d3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.327892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.328566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.328813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.329124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.336580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.338062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:26.350426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:26.351674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 29 14:17:26.353171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:26.356334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 318/643] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:26.357215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.419684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7881a9-5ae2-4929-b3fb-b99ec52f99d3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.438231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.442495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.455771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.456417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.456691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.456967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.464615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.465392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001560-06b8-47a6-bdc3-dbbcce0f5692 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.466191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-16e2ede6-76a2-4771-9368-c88ab201c8db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:26.478152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:26.479261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:26.480553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:26.484188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 326/644] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:26.485526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.485814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.486091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.505847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:26.506031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.506643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.506923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.507027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:26.507244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.518716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.520175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:26.535160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:26.536100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:26.537072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:26.537072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:26.537072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:26.537072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:26.537072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 319/645] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:26.537947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.585160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 327/646] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:26.589931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.654240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f21af-fff1-4943-9267-448c046efc8c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.686196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.691849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.692475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.692758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.693074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.693873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.695927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.696128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.701028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d471e3f-6135-464d-beba-1652858f4af5 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.712690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.715006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 320/647] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.715833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.742706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.743336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.743617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.743894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.744648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.744958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.746434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.758178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 328/648] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.758978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.839886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02628c-c2f3-431d-b3b1-f5826c994f81 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.870458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cafe3-f7b4-450a-8a8d-5ab488258e5d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:26.879952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.880566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.881088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.881216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.881826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:26.882814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:26.884222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:26.906935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 321/649] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:26.906935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.920759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:26.921580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:26.921867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:26.922146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:26.936911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:26.942145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:26.943061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:26.943903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 329/650] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:26.944702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:26.975617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:26.976908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.033745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d999f5-fc13-4d34-a9de-072167c99d4b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.047699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e1e0-740c-42d8-b71c-891e4537fef3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.073723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.074489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.075100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.075386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.083293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.084307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.084598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.084877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.086244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:27.086898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.088674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.105376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 330/651] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:27.106184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.154952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.156596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.226642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.232648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.242547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49ae98-5913-4ea7-97cf-1424d70d1999 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.281082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.282060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.282937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.283379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.293911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.295681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.299247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 322/652] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:27.300153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:27.309534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:27.310734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 331/653] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:27.311902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.433951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5bb11-bf83-4206-873b-9c992f6f459f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.434991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115b691-d991-4b7a-a690-57043c8a26ac None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.470065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.471067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.471542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.472006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.473042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:27.473567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.475378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.477104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.477834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.478074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.478350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.490129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.490475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 323/654] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:27.491771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.492003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.505221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:27.509542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:27.510424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 332/655] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:27.511987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.515826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.605834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a7028-fc1c-4aa1-80fa-5feb7ed71f51 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.625937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a1a7-6d59-4b96-b1f5-baec14d783cb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.640967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.641918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.642415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.642881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.643859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:27.644359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.645991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.664133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.664784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.665091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.665422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.667409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 324/656] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:27.668590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.677267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.678839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:27.695747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:27.696691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:27.697596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:27.697596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:27.697596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:27.697596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:27.701565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 333/657] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:27.702446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.748331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.766826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.785613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e16c3-22f4-44fb-b86c-cbcf76cacee8 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.815212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b243ce-8ff9-4505-8212-a66f73cdcd30 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.823875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.824892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.825409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.825951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.826815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:27.827261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.828761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.842633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 325/658] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:27.843701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:27.854618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.855360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.855640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.855914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.928505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.930476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:27.954995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1375-a5d4-4cf3-94f2-02df14126f7d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:27.983533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.988294 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:27.992124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:27.996075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda9c81-c5f6-4b01-b594-fc0d80c75d72 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 326/659] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8b2d8-e566-4308-ba35-8c481e3ffb0a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8b2d8-e566-4308-ba35-8c481e3ffb0a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8b2d8-e566-4308-ba35-8c481e3ffb0a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 327/660] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:28.012582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8b2d8-e566-4308-ba35-8c481e3ffb0a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.043329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 334/661] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:28.043329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.120484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8b2d8-e566-4308-ba35-8c481e3ffb0a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.142719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51672a5-5abf-4ee4-b916-c1e84fa60668 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.157964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.158697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.159067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.159410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.167103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.168547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c1b08ae8-717e-4912-be8f-56dd81cc28e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:28.179897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:28.180974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:28.181899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 328/662] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:28.182424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.182880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.182880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.189677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.190323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.190603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.190879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.198863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.200310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.201698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:28.202889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:28.212545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:28.213634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 335/663] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:28.214593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.233450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.254445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.287966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 329/664] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:28.287966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.334203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b652ea1-35ee-4e95-913b-3dd4f9e193ed None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.370683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.371629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.371629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.371885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.385030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.386526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.388910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e7c8-312d-479b-a030-101a342bc051 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:28.401618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:28.402568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:28.403488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:28.403488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:28.403488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:28.403488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:28.407316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 336/665] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:28.408225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.428733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.430232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.430505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.430778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.431541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:28.431843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.433289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.446590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 330/666] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.447366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.488234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.490071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.523819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a7298-501b-4917-bc6c-1db8378a61be None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.549386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8cc57-5741-4de2-a082-48ea0231a404 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.560779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.561543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.561919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.562251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.563116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:28.563484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.565032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.586903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 337/667] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.586903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.587170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.587293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.587493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.587768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.598007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:28.603370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:28.604320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:28.605217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 331/668] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:28.606501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.698251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9c01-aaa2-4f0a-ace1-515af5b2289d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.706394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.711731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83f08e-0322-42ee-82b4-95d738f15ffb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.726370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:28.735111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.735729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.736009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.736349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.736967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:28.737313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.738701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.746414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9269278-d728-4696-a47e-248056433693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.747154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9269278-d728-4696-a47e-248056433693 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.747439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9269278-d728-4696-a47e-248056433693 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.747709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9269278-d728-4696-a47e-248056433693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.748523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9269278-d728-4696-a47e-248056433693 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=a86fe5edd283422cb3d5650845927693) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:28.748807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9269278-d728-4696-a47e-248056433693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.750247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9269278-d728-4696-a47e-248056433693 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.755056 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 338/669] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.755056 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.760888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 332/670] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/role_assignments?user.id=a86fe5edd283422cb3d5650845927693&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.762410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9269278-d728-4696-a47e-248056433693 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:28.879449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31080e03-8b70-4503-88f8-74e3fe5feee5 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.907336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9269278-d728-4696-a47e-248056433693 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:28.936305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.936930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.937236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.937512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.954981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:28.955608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:28.955886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:28.956157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:28.956905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:28.957309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:28.959255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:28.973300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 333/671] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:28.974291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.005985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.018930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.038795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.040537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.108274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9b49-ec83-4ec7-853a-08d5e1df2cd9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.153902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 339/672] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:29.155275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.165922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.166587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.166919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.167192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.176427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.178367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:29.193597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:29.194699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:29.195953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 334/673] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:29.200881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.261226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a715cb-56d7-4eca-9698-73bcb8eddc9a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.299622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.300786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.301427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.302892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.303617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.304862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.316523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.318031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.324325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b436653-2352-4648-b494-7a6580bfd6e5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:29.334536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:29.335910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:29.337049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:29.337049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:29.337049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:29.337049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:29.338481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 340/674] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:29.339433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.370006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.370926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.371592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.372236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.384000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.385602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6154e111-4813-43af-af05-5715f42f4ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:29.400716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:29.401861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:29.402970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:29.402970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:29.402970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:29.402970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:29.404211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 335/675] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:29.405440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683e5-819f-4f39-b09f-b91e490d69dd None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.405824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683e5-819f-4f39-b09f-b91e490d69dd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.406197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415683e5-819f-4f39-b09f-b91e490d69dd None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.409248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 336/676] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:29.410054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415683e5-819f-4f39-b09f-b91e490d69dd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.456382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464319-b27e-4e65-bacd-2461dfa53e3e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.492313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.492926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.493229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.493598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.494341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:29.494645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.496071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.503899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.514405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 341/677] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:29.515282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.523132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415683e5-819f-4f39-b09f-b91e490d69dd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.561595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.562650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.563152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.563813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.564833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:29.565402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.567204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b78246-2db1-45b3-95ed-feafe48520c2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.570827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.581066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 337/678] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:29.585430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.585430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.585430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.600261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:29.603135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:29.629004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd35a5-98a8-4532-9b0e-ee9573f58b23 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.666584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.667127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.667408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.667681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.668369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:29.668681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.670085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 338/679] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:29.677033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.696284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 342/680] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:29.698564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.763793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.785082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98be7a-e9bb-45ac-a053-1e8ffc43a465 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.807777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773be89-b50c-401f-87f2-ee5012d470a4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:29.814035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:29.823168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=c7c0f85f0b514af0bc2de51aeaf4ffdf) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.829446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.845045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 339/681] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/role_assignments?user.id=c7c0f85f0b514af0bc2de51aeaf4ffdf&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:29.845045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.857423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:29.858183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:29.858538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:29.858886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:29.869992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:29.871851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:29.887881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:29.889059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:29.890114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:29.894799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 343/682] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:29.895772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:29.977910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182ee2-4c2b-4a45-a76a-946f6ce9b3d1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.018208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.019020 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.019395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.019767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.029764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.031232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.044514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf8d2d-ed42-4ea2-ba4d-98d70b1e71a6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:30.047989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:30.048853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:30.049944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:30.053903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 340/683] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:30.056104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.068273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.101017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.101017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.101017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.101017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.115869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.116038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:30.123318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:30.124353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:30.125223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:30.130485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 344/684] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:30.133020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.180639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed645a-c170-4624-86d1-2b19a8a3b4cc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.218557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.219342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.219632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.219965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.220924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:30.221280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.222714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.243233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 341/685] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:30.244176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.266254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfed83-b294-453f-b594-b34963148d1a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.301705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.302327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.302612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.302889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.303637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:30.303967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.305738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.318655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 345/686] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:30.319493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.363121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.379736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44c35b-3d13-4e54-ac00-4ebdf6243f18 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.393301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.426065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.427182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.427983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.428273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.429629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:30.429920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.430167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13275b4-4e77-417c-9cce-68326df26037 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.431308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.449146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 342/687] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:30.450046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.476574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.477288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.477576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.478121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.554478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.555956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.576531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2199f8-bcb8-472c-b7c9-4c5eefbfd560 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.608377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.617933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.618915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.619439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.619892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.620949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=06cce26ee600472f84a9d8f7b782bdb0) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:30.621445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.623111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.637188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 343/688] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/role_assignments?user.id=06cce26ee600472f84a9d8f7b782bdb0&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:30.638471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.683771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 346/689] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:30.692283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.713373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.851509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbe8e1-037e-4527-9539-b56aff8ff2bc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.891783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7fd48-1707-4503-95fd-8090d962bdb4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:30.894350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.895080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.895270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.895548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.907614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.909116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:30.930567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:30.931544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:30.932464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:30.932464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:30.932464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:30.932464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:30.932464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 344/690] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:30.938798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:30.938954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:30.939537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:30.939814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:30.940085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:30.949558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:30.956133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:30.957658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:30.973880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:30.980389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 347/691] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:30.982389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.069266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.090464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a609c6-1d51-44b7-bb1d-b33ee52d6b7f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.100416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9affa-49ca-4ec3-97aa-b989a3275ef4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.139716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.140258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.140533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.140802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.141550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:31.141829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.143186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279705a-13a1-4d82-b323-bb34e4489093 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.144198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.145271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.146219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.146718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.147616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:31.148826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.154750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.164019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 345/692] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:31.165510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360c11-b8de-4d15-8f37-5f1e0144c233 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.165795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360c11-b8de-4d15-8f37-5f1e0144c233 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.166068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb360c11-b8de-4d15-8f37-5f1e0144c233 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.167988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 346/693] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:31.172436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.172892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.173201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.174145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 348/694] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:31.179410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.195241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:31.196577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:31.269392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.276454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 347/695] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 29 14:17:31.277516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.316797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78347328-9250-4bed-9e97-223e325d78ec None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.353990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.354634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.354916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.355190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.363270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.364386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.371101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:31.377886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:31.379033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:31.380023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 349/696] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:31.380624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.402389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282f98-8eec-4b72-b630-b7793e2972c6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.446969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.448482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.448958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.449460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.458363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.460084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:31.475951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:31.477444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:31.478321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:31.478751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 348/697] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:31.484067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.503822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f62a8-c99c-4942-abf9-d36ff0bf550e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.520122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.550872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.552449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.552758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.553065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.566972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.568508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:31.584220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:31.585237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:31.586462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:31.586462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:31.586462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:31.586462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:31.586462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 350/698] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:31.586751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.606899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa30c-68b6-44f9-93cd-93c8aec8cf26 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.648479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.649223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.649604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.649979 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.650880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:31.651292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.652984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.656808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.674069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 349/699] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:31.675637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.715179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645efce-2a99-401f-adb3-dfbb059f7cf3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.758339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.758661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.759048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.759474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.760326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:31.760735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.762351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.780540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 351/700] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:31.780540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.801969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cfd43-52bf-47ed-a78f-c7a343b292c5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.806936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.852440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.853605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.854058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.854507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.855955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:31.856702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:31.858530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:31.875115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 350/701] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:31.876170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:31.903351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f85d-8d99-4db2-af3b-1f09840b55b2 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:31.943388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:31.949188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:31.949901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:31.950278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:31.950627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:31.998315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee46ab-1e81-42be-9fb0-280bda4dbe96 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.027153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.028761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9070a-452d-4a62-9248-9250fb9bbecf None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.034109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.034720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.034995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.035263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.036040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=3189acc363fe42339aedae0a604c8b42) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:32.036341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.037764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.053273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 351/702] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/role_assignments?user.id=3189acc363fe42339aedae0a604c8b42&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:32.053273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.066912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.171796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 352/703] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:32.173268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389627fb-4772-4d43-82e7-82b2f45ef329 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.173555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389627fb-4772-4d43-82e7-82b2f45ef329 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.173831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389627fb-4772-4d43-82e7-82b2f45ef329 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.175547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 353/704] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:32.177028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389627fb-4772-4d43-82e7-82b2f45ef329 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.206293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbfba1-220d-4ef0-8b5a-1a111d3294f7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.225890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.251123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.252499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.252752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.253050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.253729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.255913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c9638-6d21-4726-985d-ad8eb8a354df None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.305812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389627fb-4772-4d43-82e7-82b2f45ef329 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.331221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.343912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 352/705] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:17:32 2022] POST /identity/v3/users => generated 254 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:32.345662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.346170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.346632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.354608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.355427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.355713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.356042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.369477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:32.379872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:32.380191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.380283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:32.408124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:32.409136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:32.410413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:32.410413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:32.410413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:32.410413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:32.410413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 354/706] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:32.415061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.482363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 353/707] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:32.541496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.582660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce70ed-b79a-413a-a890-7f42eba10d55 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.624037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.624765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.625074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.625352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.626049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:32.626350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.627838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.643523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 355/708] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:32.648960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.659189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7b569-c774-4232-b691-d9bced48d58a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.661494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.687900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.698728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.699795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.700284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.700830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.701710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.703337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:32.717822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:32.718904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:32.719804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:32.722837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 354/709] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:32.730166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.788528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3e58-2126-4a74-a384-284bd91fb9cb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.835844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.836677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.837595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.837595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.847021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.849122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.861885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ab43-620d-49b2-b35b-f601513a582e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:32.865873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:32.867650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:32.869361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 356/710] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:32.873209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.909976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:32.910528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:32.910800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:32.911133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:32.911882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:32.912189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:32.915136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:32.924449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 355/711] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:32.926076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:32.959555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.991169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:32.999946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeab20-b6db-4440-82fe-e053d5c2138f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.040531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.041647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.041830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874830fc-08dd-4e65-9dd2-6ae82e3d816c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.042326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.042914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.044020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:33.044534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.046290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043523c4-9e3f-4230-a75e-6d5757513a38 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.060729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 357/712] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:33.062475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b066-16dd-48b8-892c-b17fc2d98cbe None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.064138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b066-16dd-48b8-892c-b17fc2d98cbe None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.064636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b066-16dd-48b8-892c-b17fc2d98cbe None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.066613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 358/713] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:33.067714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99b066-16dd-48b8-892c-b17fc2d98cbe None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:33.078038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.078956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.078956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.079230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.079886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:33.080187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.081582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624928de-87d7-430f-bf36-650af4f0b98d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.092623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 356/714] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:33.092623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.092623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.092623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.112428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:33.113733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:33.199549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 357/715] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:33.229871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:33.273542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:33.282956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99b066-16dd-48b8-892c-b17fc2d98cbe None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.325107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.325912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.326197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.326542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.327443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:33.327822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.329507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.336341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 359/716] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:33 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:33.371677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa5c0-2b90-482a-a60a-6516f98cbbe8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.411475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.412454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.412919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.413976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.415072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.416755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:33.427078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:33.428159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:33.429059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:33.432298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 358/717] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:33 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:33.442417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:33.479628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:33.597248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c6e4-f3bd-4ec0-820f-34bf639137c6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.639120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.639789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.640066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.640387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.641084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:33.641384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.642749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6c4e9-49cb-4e96-a6e0-89ce86a932f2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.648295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 360/718] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:33.653561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:33.792284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:33.802033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834725-80b5-4235-8f8e-1a4487e3c4e1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:33.847247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:33.847871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:33.848150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:33.848427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:33.849144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] RBAC: Adding query filter params (type=network) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:33.849452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:33.850817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca89b-cf08-4acf-88e5-9b7ed9be13ca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:33.856157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 359/719] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:34.546163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55579e-fe99-4dd4-81a5-33e1ea3e5357 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:34.546720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55579e-fe99-4dd4-81a5-33e1ea3e5357 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:34.546720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55579e-fe99-4dd4-81a5-33e1ea3e5357 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:34.548321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 361/720] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:34.553337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:34.553861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:34.554408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:34.582041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:34.583570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:34.679833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 360/721] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:34.714381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55579e-fe99-4dd4-81a5-33e1ea3e5357 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:34.860027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:34.891152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55579e-fe99-4dd4-81a5-33e1ea3e5357 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:34.939572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:34.940226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:34.940510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:34.940790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:34.957842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-dbc49d89-5437-42e6-8a63-1f304d87ec93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:34.963329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:34.964270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:34.967118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 362/722] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:34.970330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.042516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d874-7d57-44a0-b486-f29e3b2d3cba None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.042824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d874-7d57-44a0-b486-f29e3b2d3cba None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.043085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d874-7d57-44a0-b486-f29e3b2d3cba None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.044695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 363/723] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:35.051995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.052284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.052561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.066713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.084984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127b617-c9f0-481b-a2a8-525c9395ae23 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.089984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:35.091189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:35.124335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.124931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.125248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.125579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.126310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:35.126618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.128072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.138144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 361/724] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:35.143704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.252771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8b25-fb24-4550-a506-44eafdfdfc10 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.290202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517acee-06df-42ef-9867-427dad057228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.290848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517acee-06df-42ef-9867-427dad057228 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.291130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517acee-06df-42ef-9867-427dad057228 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.291450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517acee-06df-42ef-9867-427dad057228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.303065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517acee-06df-42ef-9867-427dad057228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.305574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517acee-06df-42ef-9867-427dad057228 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-0517acee-06df-42ef-9867-427dad057228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:35.321285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:35.322304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:35.323601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:35.323601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:35.323601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:35.323601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:35.323601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 362/725] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:35.324481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.327013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 364/726] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:35.328198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517acee-06df-42ef-9867-427dad057228 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.370462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.465323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.508833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab92755-227f-4b40-8d3d-7c86551a010b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.573301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517acee-06df-42ef-9867-427dad057228 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.573723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.574397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.574687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.574962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.594169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:35.607244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:35.608213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:35.609495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 365/727] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:35.611425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.612052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.612341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.612614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.613354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:35.613664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.615157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.621957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.628394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 363/728] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:35.634522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.697496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.743300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba95524-7ae1-4aee-900e-a0b55fe1c70c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.779331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.779989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.780274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.780552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.789723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.791511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:35.804139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:35.805326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:35.806916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 364/729] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:35.812388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.876098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aad6-49c4-42b4-8053-8d23cd3d723a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.916970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.923092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9a0-112b-4563-8f42-44b6f5f95d67 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:35.923529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.924152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.924431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.924706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.925464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:35.925971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.928373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fb98a-6ff8-48e2-b5e6-926c6263dc5e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.938740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 366/730] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:35.941046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93711257-ed8c-4ff3-b11d-325a0666b235 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.941337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93711257-ed8c-4ff3-b11d-325a0666b235 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.941617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93711257-ed8c-4ff3-b11d-325a0666b235 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.947339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 367/731] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:35.948561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1578-0b0d-484e-93b2-0fa8d508645e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.948852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1578-0b0d-484e-93b2-0fa8d508645e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.949313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1578-0b0d-484e-93b2-0fa8d508645e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.949641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:35.953526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 368/732] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:35.953526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1578-0b0d-484e-93b2-0fa8d508645e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:35.968309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:35.968935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.969242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.969524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:35.970317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:35.970630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:35.972056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd559313-0c00-4a8b-8ff9-73b84980227f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:35.993521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 365/733] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:35.994718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a02bb-295f-462b-b308-bf8a4c7fab25 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:35.995004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a02bb-295f-462b-b308-bf8a4c7fab25 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:35.995279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a02bb-295f-462b-b308-bf8a4c7fab25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.013711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b02a02bb-295f-462b-b308-bf8a4c7fab25 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:36.014891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b02a02bb-295f-462b-b308-bf8a4c7fab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:36.067002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1578-0b0d-484e-93b2-0fa8d508645e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.086499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 366/734] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:36.088088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.088556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.088835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.110031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:36.111665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:36.117416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.118401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.118882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.119384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.132072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.133756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:36.150088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:36.151033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 369/735] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:36.156748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.188836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 367/736] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:36.201892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.210756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.239651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.269685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25075cd1-4699-4d37-9b24-a19e2eca5110 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.305565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.306189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.306474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.306752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.307808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e60de0-a91e-47ad-933c-d149822c7461 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.347215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.347838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.348119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.348392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.349507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.350909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.377597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.379063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.425657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.447950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 368/737] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:17:36 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:36.448916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.484362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 370/738] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 29 14:17:36.485518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.565291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.587373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c493be2-5868-4090-9c4d-b15aa0f2529d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.603734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1024a-e4b4-449b-9ed1-9e75537abd96 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.630399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.631282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.631565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.631839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.632717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:36.633095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.634588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c512f4-972f-4961-bed9-0d16c44add8b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.645474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.646116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.646396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.646672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.646776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 369/739] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:36.648633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820d52f-1101-45e3-a95f-db63c3feb59c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.649397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820d52f-1101-45e3-a95f-db63c3feb59c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.649690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820d52f-1101-45e3-a95f-db63c3feb59c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.651536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 370/740] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:36.652403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820d52f-1101-45e3-a95f-db63c3feb59c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.658706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:36.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:36.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:36.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 371/741] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:36.666906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.676167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.765773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820d52f-1101-45e3-a95f-db63c3feb59c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.773284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cad24-a149-4983-a971-0064a3ef4178 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.806570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.807199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.807470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.807739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.810271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.810908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.811189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.811482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.818784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.819364 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.820792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.821466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-3645fb29-54d9-4f91-97a6-cfc8383ca139 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:36.832917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:36.833914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:36.834857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 372/742] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:36.835988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a753a-f128-457d-90db-b733e8f95084 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.836184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a753a-f128-457d-90db-b733e8f95084 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.836470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a753a-f128-457d-90db-b733e8f95084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:36.841360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:36.842224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 371/743] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:36.844582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:36.848389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:36.859474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-454a753a-f128-457d-90db-b733e8f95084 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:36.869345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-454a753a-f128-457d-90db-b733e8f95084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:36.955153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85e305-359b-472c-a4ab-623c75d1bcfd None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:36.993744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:36.994435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:36.994665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:36.994912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:36.995645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:36.995945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:36.997379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.011177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 372/744] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:37.011177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.042918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.050534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 373/745] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 29 14:17:37.051404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a753a-f128-457d-90db-b733e8f95084 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.090436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.130259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991133c-30a1-4f5b-a91b-b7a037e019ee None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.170191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a753a-f128-457d-90db-b733e8f95084 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.171547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.172179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.172460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.172739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.173763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:37.174077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.175505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.193451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 373/746] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:37.194294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.215824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.217043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.217532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.217986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.218858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:37.219764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.221597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.241867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 374/747] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:37.241867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.294474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.309068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23c8f-008c-41c9-aa5a-cb8ea1b0d5c7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.334003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.339649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93019a-08db-417f-b0f0-1c33c771f9ae None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.348932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.349791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.350537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.351401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.362460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.365562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.374480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.375096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.375380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.375657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.376346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.377732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:37.382332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:37.383377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:37.384278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:37.384278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:37.384278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:37.384278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:37.384278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 374/748] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:37.385038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.461024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 375/749] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:17:37 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:37.461024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.502685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9b91-96d9-49ee-958a-1a708014da3a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.547446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.548290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.548639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.548984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.574082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bae5c-5c7e-4886-9e3b-1ea1b54310ff None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.596278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.599597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.620400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.621165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.621378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.621659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.629497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.631699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.636900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.638694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:37.646464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:37.652709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 376/750] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:37.653658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.763286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191b176-35d1-469e-80e6-945f0ca8f588 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.770390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 375/751] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:17:37.771292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.801638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.802748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.803340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.803822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.805356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:37.805876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.807597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.821533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 377/752] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:37.826868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.845922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.876190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb8911-21e5-4159-8ff9-7776dbbf3ecc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.877376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:37.912836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.913592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.913929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.914390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:37.915141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:37.915450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:37.916879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:37.929819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 376/753] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:37.930840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:37.944592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd9be8-fe10-4eab-a336-7a028c37fa1f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:37.984567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:37.985615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:37.986208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:37.986737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.000374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.001998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:38.025220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:38.026172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:38.027057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:38.027057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:38.027057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:38.027057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:38.029997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 378/754] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:38.031169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.046055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6434e-620c-41f5-8457-3346242a72f5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.084106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.092164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.093447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.094724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.094724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.104917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.107293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:38.123023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:38.123999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:38.124949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 377/755] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:38.125568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.148915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.167545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdac18b-4138-4b5b-99f0-b3ab375e5c4c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.204724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.205581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.206428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.206428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.207282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=3189acc363fe42339aedae0a604c8b42) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:38.207683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.209887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.224106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 379/756] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/role_assignments?user.id=3189acc363fe42339aedae0a604c8b42&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:38.225226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.244678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339f4f5-8a3b-4a4e-9c8e-a7610d4de898 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.282862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.283730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.284156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.284576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.285578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:38.286092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.287719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.307471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 378/757] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:38.309215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.368471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.408619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec078e85-04c0-41e7-8ea9-da017c6535e9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.420799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be466d7-450c-43a7-95ed-008c4e3e2b9d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.459186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.460286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.461019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.461565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.469820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.471745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:38.485523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:38.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:38.487404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:38.495214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 379/758] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:38.496940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.509906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.510448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.510731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.511003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.511825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:38.512133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.513638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.542350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 380/759] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:38.547838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.562247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.630654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26e916-ba62-4349-a5db-e612970985ad None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.648895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.667190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f55077-1768-4fd3-92f5-c6d68bd75c43 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.672778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.673495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.673777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.674056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.675025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:38.675337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.676907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbf5af-37ed-4ae9-8079-2b1404325b78 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.694953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 380/760] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:38.696347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad2ad-ddbe-40cc-a62c-e2b9e0c2b2b7 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.696636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad2ad-ddbe-40cc-a62c-e2b9e0c2b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.696914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad2ad-ddbe-40cc-a62c-e2b9e0c2b2b7 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.698751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 381/761] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:38.701878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ad2ad-ddbe-40cc-a62c-e2b9e0c2b2b7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.720459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.721574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.722007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.722932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.739184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.741120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-a7e78ef0-5245-43f2-9424-38b041a49a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:38.765858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:38.767258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:38.769110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:38.769110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:38.769110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:38.769110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:38.772951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 381/762] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:38.774752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.775314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.775600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.799953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:38.801956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:38.822357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ad2ad-ddbe-40cc-a62c-e2b9e0c2b2b7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:38.870911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:38.871879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:38.872158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:38.872439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:38.873403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=b4710cdb68364a5696a2e33ad6232650) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:38.873717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:38.875706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:38.878931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:38.885641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 382/763] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/role_assignments?user.id=b4710cdb68364a5696a2e33ad6232650&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:38.898327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.898472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 382/764] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:38.916673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:38.977196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.050117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270caaaf-83a7-4b20-ae57-937b34be7324 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.090431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72985d0-751d-422d-a1a7-68a99a97a80b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.093625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.094390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.094673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.094951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.095585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.096973 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:39.110654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:39.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:39.112475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 383/765] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:39.119008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.152718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.154178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.154668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.155172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.156095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:39.156600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.158869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.175134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 383/766] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:39.180661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.242645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.299389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad6a3-2f0d-4ddc-acba-dac3095af65a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.336084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23764467-b377-4933-b896-699c0ac96984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.337110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764467-b377-4933-b896-699c0ac96984 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.337591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764467-b377-4933-b896-699c0ac96984 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.338078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764467-b377-4933-b896-699c0ac96984 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.338958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23764467-b377-4933-b896-699c0ac96984 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:39.339487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23764467-b377-4933-b896-699c0ac96984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.341104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23764467-b377-4933-b896-699c0ac96984 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.347049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 384/767] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:39.356233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23764467-b377-4933-b896-699c0ac96984 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.365530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.372365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbcc59-0dd2-446b-a1c4-4db94d33f2db None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.458986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.459649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.459934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.460212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.478283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.487758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23764467-b377-4933-b896-699c0ac96984 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:39.506354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:39.507330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:39.508236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 384/768] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:39.513008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.522126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.551062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.551687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.551968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.552248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.552940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:39.553277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.555636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.571224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 385/769] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:39.578012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.681269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74047b1-d9fd-4632-8907-3dcdc1d12874 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.722719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a20fa3-748b-448e-83cd-083dc1c63241 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.730552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.732091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.732629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.733145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.734589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:39.735115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.737479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738b3c6-3245-4863-b917-56bf25c609af None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.755534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 385/770] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:39.757468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3cade-3d94-4cfc-b540-218562e12cbd None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.757960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3cade-3d94-4cfc-b540-218562e12cbd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.758449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3cade-3d94-4cfc-b540-218562e12cbd None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.760153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 386/771] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:39.763174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3cade-3d94-4cfc-b540-218562e12cbd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.767284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.767921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.768203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.769105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.772385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] RBAC: Adding query filter params (interface=public, service_id=c65180c59a5d4a2b906d2c4d8c9bded7, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:39.773511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.775183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9d96a-f37c-49d7-9ffc-4e1fd24dee70 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.784615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 386/772] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/endpoints?service_id=c65180c59a5d4a2b906d2c4d8c9bded7&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:39.786193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.786689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.787166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.790767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.808318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:39.819276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:39.859077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:39.880495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3cade-3d94-4cfc-b540-218562e12cbd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:39.885912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 387/773] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:39.886950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:39.927687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:39.928444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:39.928805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:39.929761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:39.931812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=e4a3cbcc38d741178be408abd4156959, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:39.933475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:39.996830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc318491-6b7a-4e84-b930-49ec8dd8f999 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:40.025176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 387/774] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:39 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:17:40.026320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:40.034828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:40.035720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.036147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.036569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.037398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:40.039595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:40.046288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 388/775] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:17:39 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:40.165020 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:40.197054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1f02-493b-4ac1-84be-46fe753f5df6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:40.256975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:40.258238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.258508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.258784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] PATH_INFO: `/v3/services/network` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.259449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:40.261026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:40.272068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:40.273825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:40.276292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 388/776] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:40 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:40.280032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:40.338478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:40.399824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa908-10ad-459f-9363-b31560d7945a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:40.440456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:40.441114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.441400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.441677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.442363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] RBAC: Adding query filter params (name=network) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:40.442668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:40.444043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:40.449503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 389/777] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:40.455035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:40.573606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e42a0-02ea-43af-abd4-a560b0541f17 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:40.612099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:40.612724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.613017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.613297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.613980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:40.614285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:40.615649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c069b1-4d95-4063-84d8-5e1dffed68d7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:40.621714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 389/778] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:40.627621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:40.735382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab4c2-509d-423b-be0e-389ff01db280 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.735712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab4c2-509d-423b-be0e-389ff01db280 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.735928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab4c2-509d-423b-be0e-389ff01db280 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.737527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 390/779] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:40.743470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.744101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.744580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.748734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15241b23-b014-4658-98af-66c3a3f74254 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:40.756760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:40.773014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:40.778927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:40.787695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:40.792291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:40.792974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:40.793279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:40.793555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:40.794375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] RBAC: Adding query filter params (interface=public, service_id=3d01635072c842f59ccb0a082e24a0eb, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:40.794690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:40.796073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:40.801962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 390/780] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:17:40 2022] GET /identity/v3/endpoints?service_id=3d01635072c842f59ccb0a082e24a0eb&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:40.808058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:40.943000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 391/781] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:40.953659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be42e08-3c3e-4567-b512-29b53fd54d72 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:41.004531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.006609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:41.007935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.008207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.008589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.013016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:41.013016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:41.017238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 391/782] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:17:40 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:41.146713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecf7ba-6bca-49bc-b4ef-0f721b3a70ad None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:41.187561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:41.188552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.188552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.188844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.191014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:41.207554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e736a0ed-20a3-4c45-86cd-3412c994b8ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:41.213953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:41.218769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:41.220830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 392/783] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:41.225019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.288175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:41.389972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a0ed-51aa-4383-ba0c-7bd7a842a255 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:41.405078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92c33-969b-42f1-9c6c-c003269e9a71 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.405607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92c33-969b-42f1-9c6c-c003269e9a71 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.405607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92c33-969b-42f1-9c6c-c003269e9a71 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.407257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 393/784] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:41.412134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.412419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.412698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.437505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:41.439452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:41.446384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:41.447028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.447314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.447590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.448327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:41.448645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:41.450575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:41.462676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 392/785] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:41.469978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.560524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 394/786] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:41.588483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.674938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:41.690969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ffb4-0d2a-49fd-901b-04688b3ce232 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:41.712666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a462d-958c-4b2d-8ec6-e3a676b3da5b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:41.742376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:41.755380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:41.756062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.756348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.756627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.771492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:41.777575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:41.780703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:41.783354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 395/787] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:41.789683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.795827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:41.797091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:41.797519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:41.797960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:41.817560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:41.820909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:41.838215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:41.839210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:41.840122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:41.840122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:41.840122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:41.840122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:41.842397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 393/788] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:41.849696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:41.974258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580338-7393-4f6a-9fcc-b83243f81dff None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.021126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.022103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.025830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.026347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.027282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:42.027808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.029482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.045333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 396/789] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:42.047015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.072014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475352ed-24eb-4005-adb2-87120f2c8310 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.097361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.152491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.153523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.154034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.154521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.155788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:42.155788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.165120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.187414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 394/790] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:42.190773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.195838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776bf9-8a30-412d-adab-6a75ab70252e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.212882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.264940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.265673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.265956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.266227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.284559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.285956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:42.314604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:42.315726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:42.316642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:42.316642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:42.316642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:42.316642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:42.316642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 397/791] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:42.325990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.352752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f6ef8-5c6f-4c82-b3b1-135efacca331 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.398344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.398995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.399274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.399546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.420029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.421540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:42.448006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:42.449252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:42.450416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:42.451064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 395/792] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:42.456512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.540102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74465e53-7e5d-44d6-a4a1-b309eb6aa7b2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.543179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.575344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.605561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.606304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.606469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.606790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.607511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:42.607808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.609315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc1784-c382-4be2-a0f4-494043905684 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.619852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f630-b749-4d5e-8f13-3c19b4133a34 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.622539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 398/793] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:42.623893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c3a8-0945-448e-8ad1-4033c2b7daab None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.624245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c3a8-0945-448e-8ad1-4033c2b7daab None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.624585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c3a8-0945-448e-8ad1-4033c2b7daab None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.629927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 399/794] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:42.629927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3c3a8-0945-448e-8ad1-4033c2b7daab None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.680871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.681541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.681820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.682099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.682905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:42.683214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.684909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f702df-1659-4564-b861-44b96a7bbdcb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.704554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 396/795] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:42.705947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.706243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.706522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.725820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:42.727801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:42.754521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3c3a8-0945-448e-8ad1-4033c2b7daab None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.791672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.792276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.792607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.792882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.800243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.801782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.804680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 397/796] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:42.805676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1a5ac2ce-604d-4278-9647-9c82bd3756a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:42.814722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:42.815665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:42.816640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 400/797] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:42.818073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675274-bd9b-4265-97e5-88ba470878b4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.818073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675274-bd9b-4265-97e5-88ba470878b4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.818073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79675274-bd9b-4265-97e5-88ba470878b4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.825023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 401/798] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:42.825895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79675274-bd9b-4265-97e5-88ba470878b4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:42.847424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.872873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:42.922824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114837a-cde9-408d-b9d9-127337fca215 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.938863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79675274-bd9b-4265-97e5-88ba470878b4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:42.959384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.960024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.960345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.960621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.970980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.972409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.974628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:42.975245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.975526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.975796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:42.977351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:42.977659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:42.979102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-083839a2-f52c-4c52-9c9e-3ec66aeb6b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:42.987932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:42.988858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:42.989796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:42.989796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:42.989796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:42.989796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:42.994416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 398/799] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:42.995473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:42.995757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:42.996036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.004542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 402/800] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.004542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.018782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:43.019951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:43.090838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.097535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 399/801] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:43.105140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.114457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b187dd7-8a33-4004-9075-0aeb4b5dc4e7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.151230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.155771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.156426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.156768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.157103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.157902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.159110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:43.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:43.174883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:43.175715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 403/802] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Jul 29 14:17:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:43.176562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.223792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fee2f-e207-4153-80c6-a67c0cdce42b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.259383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.260010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.260288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.260558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.261300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.261606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.262969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.279209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 400/803] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.279209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.286354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb529-677a-4713-9684-bc6591d49465 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.321216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.321853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.322132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.322409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.331392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.334375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.336860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:43.352234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:43.353410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 404/804] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:43.358944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.382317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.386856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745ebe3-ca46-48b1-b254-d85286ac6c09 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.423591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.424236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.424516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.424795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.436383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:43.441777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:43.442901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:43.443875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 401/805] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:43.448554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.468751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638917-61dd-419b-a6f5-5169b17bc2ad None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.504576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.505227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.505512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.505785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.506464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.506770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.508462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.517580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 405/806] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.517580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.563367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f35530-2381-4850-81b3-f2f2bc1fc731 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.576480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.598183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.598822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.599105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.599380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.607293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.608668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.620273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a71a5d-fb89-4cb3-a3bc-bae262851a27 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:43.620829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:43.621836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:43.622763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 402/807] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:43.623264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.643043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.656026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.656646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.656925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.657228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.657925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.658228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.659590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.671993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 406/808] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.672802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.725612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928c0-93e8-4ee3-82fd-4aa8afd7c8ab None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.759567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.760178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.760457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.760728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.761472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.761771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.763131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148e299-09a4-439a-a8d4-8dd90bb1135d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.770993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 403/809] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.772134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4b5bf-d37c-412d-b512-451812879df6 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.772416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4b5bf-d37c-412d-b512-451812879df6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.772517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef79e5-ce26-4c18-8eda-40891a1bf1a1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.772691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4b5bf-d37c-412d-b512-451812879df6 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.777034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 404/810] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:43.777034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4b5bf-d37c-412d-b512-451812879df6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.792507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.811241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.811940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.812225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.812493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.813196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.813499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.814887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.819577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 407/811] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.820438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.858947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:43.879339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4b5bf-d37c-412d-b512-451812879df6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.915890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.916506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.916789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.917097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.917848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:43.918158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.919021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0dc71-ce90-46ad-b303-50d0ea25c3ae None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:43.919594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.931925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 405/812] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:43.932723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:43.958096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:43.958735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.959014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.959290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.966390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:43.967816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-232c6e24-0d7b-4f49-aafc-32379b19c002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:43.978898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:43.979928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:43.980892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 408/813] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:43.981426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:43.981426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:43.981937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:43.998642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:43.999821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:44.034229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.037261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbe558-1cf2-415e-b623-a9e1a7f6739c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.069058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 409/814] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:44.070408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.072714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.073388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.073679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.073948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.088691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.090664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.092480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:44.108029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:44.109145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:44.110038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:44.110038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:44.110038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:44.110038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:44.110038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 406/815] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:44.110325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.175810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0991f55-25ec-48e3-87ec-143d47f359aa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.209208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.209827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.210098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.210991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.211832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.213208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.216125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2eeb0-9a10-45a5-b3c5-eee01d3c02c7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.249598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.250261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.250546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.250824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/3189acc363fe42339aedae0a604c8b42/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.252759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=3189acc363fe42339aedae0a604c8b42, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.254230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.335307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 410/816] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:17:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:44.336128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.351933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 407/817] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:44 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/3189acc363fe42339aedae0a604c8b42/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:44.353183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.398565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.429989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.492838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6f85-9f98-48be-b131-2243ded5e550 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.526237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8805f-0f19-4eb3-8c70-ba52f5c9b0fe None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.542416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.543165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.543513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.543853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] PATH_INFO: `/v3/services/network` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.544560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.546043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:44.556147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:44.557457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:44.558424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 408/818] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:44.559272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.572392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.573057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.573324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.573591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.574348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:44.574646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.576067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.589241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 411/819] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:44.590140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.681743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d20033-4679-4c71-9642-96c51dbba670 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.698625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.706462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:44.718354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96001-c45a-480b-9c84-bb75d6c9364e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.731531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.732474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.733579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.734013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.735579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:44.736293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.738514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.756016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 409/820] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:44.757356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.762803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.763397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.763671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.763944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.764722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] RBAC: Adding query filter params (name=network) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:44.765053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.766422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.772712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 412/821] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:44.774026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.904319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee87f4-8c7e-42e4-baed-4bcc4f37f77f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.911810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf419fb-ef83-44bf-be12-185355062144 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:44.950374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.950969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.951297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.951577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/b4710cdb68364a5696a2e33ad6232650/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.957651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:44.958295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:44.958573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:44.958849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:44.967970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=b4710cdb68364a5696a2e33ad6232650, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.968315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:44.969471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.969773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:44.982769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:44.989862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 413/822] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:44.990813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:44.994499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.045719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 410/823] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:44 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/b4710cdb68364a5696a2e33ad6232650/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:17:45.047104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:45.087539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.098070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd8fe-13bb-4556-8d1e-1522d71ed5e6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:45.184131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f45b7-e3cb-4db3-89ab-4ce6e2c76b4c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:45.185480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:45.186331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:45.186817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:45.187268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:45.188158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] RBAC: Adding query filter params (type=network) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:45.188654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:45.190332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:45.207471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 414/824] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:45.214000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:45.229035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:45.229035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:45.229653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:45.230632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:45.231558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:45.231976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:45.234211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:45.261789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 411/825] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:17:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:45.264985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:45.313941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.351955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98456de-1c0c-49e8-981a-6cf5963c1b0c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:45.373394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.389841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9004a1-70d4-48a3-a846-6e19677a2b08 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:45.428856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:45.430314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:45.430903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:45.431250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:45.455811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:45.456545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:45.456892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:45.457256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:45.458094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:45.459494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:45.504531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:45.516169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:45.590933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 415/826] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:17:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:45.598101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.616342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 412/827] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:17:45.619033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:45.791911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e087d28-b3e6-4e4c-add2-c7abcaf29019 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:45.815946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:45.827654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:45.830829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:45.830829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:45.830829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:45.840421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:45.841914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:45.857276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:45.858245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:45.859131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:45.859131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:45.859131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:45.859131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:45.859131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 416/828] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:45.865785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:45.978740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1682-5c3e-4ac9-84a4-b6bfa2cac039 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.022276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:46.023582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.024119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.024591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.025572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:46.026095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:46.027745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:46.043370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 413/829] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:46.049304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:46.179446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a018b-ef21-4ab4-afad-ab6699f379c1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.219558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:46.220211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.220490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.220762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.228495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:46.230031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:46.242361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:46.243258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:46.244227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 417/830] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:46.249752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:46.299782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:46.359310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b9be-44c3-4e19-b2fa-21b1446a4e2d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.397552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:46.398769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.399055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.399323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.400125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:46.400436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:46.401942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94048c-ec19-4600-b14a-0d421b050d16 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:46.415178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 414/831] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:46.421024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:46.457895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:46.527330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862e16a-f912-4dfb-aa37-228013879516 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.562359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:46.562900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.563173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.563442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.564225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:46.564530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:46.565973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd9700-122a-4bb0-8fc3-acefc6e1fd2a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:46.572461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 418/832] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:46 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:46.685941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda98f-c547-4bf9-a17b-7f9e8949d3ca None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.686428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda98f-c547-4bf9-a17b-7f9e8949d3ca None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.686804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda98f-c547-4bf9-a17b-7f9e8949d3ca None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.688657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 415/833] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:46.694592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.695571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.695943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.709080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3161f5-1e92-45b5-8885-58d6975daf98 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.709433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3161f5-1e92-45b5-8885-58d6975daf98 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.709636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3161f5-1e92-45b5-8885-58d6975daf98 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.711370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 416/834] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:46.716059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:46.716853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.717175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.717346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:46.717498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.744602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:46.745805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:46.802519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 419/835] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:46.827318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:46.842056 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 417/836] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:17:46.858800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:46.864864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:46.900387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:46.938789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f357d-2354-4f91-a61b-f7b1980f3670 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.976026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5585b-3b5c-4ad1-8762-776ba192fe68 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:46.979552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:46.980261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:46.980535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:46.980810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:46.992791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:46.998100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:47.000254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:17:47.002319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:47.008754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 420/837] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:47.008754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.014417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.015122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.015410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.015683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.016324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.017731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:47.027486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:47.029749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:47.031719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:47.041017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 418/838] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:47.041017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.108370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.128216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed44af7-3358-46db-87e2-a65f71de59a8 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.139606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.143175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966350d-e4b3-4d26-9351-87282ecf324f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.167427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.168060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.168372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.169626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.170365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.170676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.172958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.184691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.185073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 421/839] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:47.185829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.186110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.186404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.187098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.187403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.188777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.197980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.199158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 419/840] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:47.205441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.303460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d9985-31ef-4475-a10b-7c9957e2e41f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.316574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bb97a-7e0d-4fc7-b82c-4eb404787f63 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.342351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.342980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.343260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.343532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.354067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.354724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.354971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.355246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.355444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.356052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.356363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.356907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.357760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b2de5-ce55-4317-a545-9ddfbbc74b2a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.369358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 420/841] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:47.371141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ead0b-5b57-47cb-bcce-4c840c59441e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.371636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ead0b-5b57-47cb-bcce-4c840c59441e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.372111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ead0b-5b57-47cb-bcce-4c840c59441e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b1483f46-9256-4499-ab35-7d4e7ac30e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:47.372216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:47.373406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:47.374303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:47.374303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:47.374303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:47.374303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:47.377068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 421/842] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:47.377445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 422/843] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:47.382460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ead0b-5b57-47cb-bcce-4c840c59441e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.384479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.384757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.385606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.394569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.412861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:47.415336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:47.421019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.504352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 423/844] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:47.524301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.526111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ead0b-5b57-47cb-bcce-4c840c59441e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.573026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.573657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.573932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.574229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.574898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.575199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.577009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.593085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 422/845] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:47.600011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.648545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eff8f9-2b23-4847-903a-f582fa6df6bd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.678474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.684003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.693272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.693272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.693272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.697286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:47.698241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:47.702249 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:47.703175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:47.704335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 424/846] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:47.711231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.723653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40061c-8c38-4076-8306-67229fe81cf1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.763665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.764407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.764694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.764969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.773232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.774937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:47.789687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:47.791926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 29 14:17:47.794202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:47.797034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 423/847] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:47.798353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.828598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61e313-c7d6-43f1-a471-e97e957be386 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.866144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.866817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.867094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.867367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.868083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.868486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.869888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.878738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 425/848] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:47.883892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.916068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd61f88-af4a-43a8-a3fe-2b7379de73d2 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.942130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:47.954779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:47.955504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:47.955792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:47.956082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:47.956872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:47.957211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:47.958653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:47.984591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 424/849] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:47.985518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:47.989246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c4abc-228c-40de-9c81-4a5e5aeb6f1e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:47.989370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.025779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.026463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.026746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.027019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.039845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.041374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:48.056817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:48.057770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:48.058655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:48.058655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:48.058655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:48.058655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:48.062949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 426/850] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:48.064194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.096148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab708-9b39-4076-85bc-19ada04a53ae None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.142277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.142904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.143182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.143451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.185071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385d016-1395-4a74-89ee-dc9dce5efe0f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.193058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.224614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.225645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.226705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.227074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.227897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:48.228300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.229810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.241353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.247501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 427/851] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:48.247690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.248777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.250500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.355296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 425/852] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:17:48.355717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cacc62-ea54-45dc-a313-1a0ddd217857 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.356387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.396207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.397340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.397904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.398355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.406420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.408145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:48.421821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:48.422746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:48.423687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:48.430762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 428/853] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:48.432211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.463241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.471201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c5ffa-156e-4821-a9d6-90dda9514b2e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.473874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.508142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.508985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.509131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.509394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.519898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.521399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:48.537283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:48.538974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:48.540296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:48.540296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:48.540296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:48.540296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:48.540296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 426/854] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:48.544820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.552462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08229-944d-402b-8e60-1f35a325c892 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.602023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.602725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.603014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.603294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.604346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:48.605102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.607021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.633169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 429/855] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:48.635190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.670396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c0ed-db8b-469b-a7b5-601cc22ecb56 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.706088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.706986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.707267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.707579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.708433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:48.708540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.709926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75d23f-5d6c-4313-bfce-1c4fd98b33a6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.722812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 427/856] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:48.723889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c540-66a2-4206-bb55-78b191f7c596 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.724175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c540-66a2-4206-bb55-78b191f7c596 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.724452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8c540-66a2-4206-bb55-78b191f7c596 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.725803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 428/857] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:48.728660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8c540-66a2-4206-bb55-78b191f7c596 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.740582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.750402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.759254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328432-d64a-4c47-b46f-c4007546596c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.798821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.799434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.799705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.799974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.838040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8c540-66a2-4206-bb55-78b191f7c596 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:48.872600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:48.873156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.873268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.873534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.873811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.874598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.880870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:48.883010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e6a17057-2e84-4dda-8871-da91aa8ba7a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:48.894739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:48.903952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 429/858] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:48.904856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:48.905480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:48.918221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:48.919406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:48.970656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:48.994049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 430/859] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:17:48.994049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:48.996659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 430/860] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:48.998445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.013403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.104079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b000-7e5a-4d62-bc97-b8d15bd42897 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.111881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b62180-1d42-439c-9d75-4696d12dc7ea None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.141630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43893490-4400-4213-bdde-f200d46d81da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.142365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893490-4400-4213-bdde-f200d46d81da None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.142647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893490-4400-4213-bdde-f200d46d81da None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.143038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893490-4400-4213-bdde-f200d46d81da None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.143794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43893490-4400-4213-bdde-f200d46d81da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.144107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43893490-4400-4213-bdde-f200d46d81da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.145588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43893490-4400-4213-bdde-f200d46d81da None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.148831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.149514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.149795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.150070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.158704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 431/861] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.159213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43893490-4400-4213-bdde-f200d46d81da None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.169124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.170604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:49.187762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:49.188942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:49.189907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:49.189907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:49.189907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:49.189907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:49.189907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 431/862] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:49.190189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.216890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.251486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.277827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43893490-4400-4213-bdde-f200d46d81da None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.300394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fc511-9927-45ec-9f90-ca1727a2328d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.318407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.319344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.319625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.320096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.320799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.322710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:49.332386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:49.333692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:49.334546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:49.338872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.339274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 432/863] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:49.339549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.339832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.339986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.340141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.340930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=3189acc363fe42339aedae0a604c8b42) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.341274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.342787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.349531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 432/864] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/role_assignments?user.id=3189acc363fe42339aedae0a604c8b42&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.350123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.479791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3584701-0c47-4e9f-9f69-b112370f945b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.498423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03847-fc70-4412-9098-adbba62a2777 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.506176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.513720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.520214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.520931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.521313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.521662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.522434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.522834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.524391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.533846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 433/865] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.536050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.547370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.548439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.551705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.551705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.551705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.551705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.552145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6251f07-a696-4d43-ae6b-54bf67693826 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.566994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 433/866] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.573088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3c68e-f954-4597-a161-ca532278886a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.573380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3c68e-f954-4597-a161-ca532278886a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.573658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3c68e-f954-4597-a161-ca532278886a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.575236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 434/867] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:49.576559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3c68e-f954-4597-a161-ca532278886a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.665467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825fa4-9e9e-4a97-9a77-1be6e1e81dcc None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.686808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3c68e-f954-4597-a161-ca532278886a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.702235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.703038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.703330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.703609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.704286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.704584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.706055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dd33e-3a65-47da-b07f-8199476acdc3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.721022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 434/868] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.721022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.721022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.721022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.736622 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781051b-e64e-4341-99de-a349552e702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.737401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781051b-e64e-4341-99de-a349552e702d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.737681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781051b-e64e-4341-99de-a349552e702d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.737956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781051b-e64e-4341-99de-a349552e702d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.742581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:49.744368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:49.747131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781051b-e64e-4341-99de-a349552e702d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.749037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781051b-e64e-4341-99de-a349552e702d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-7781051b-e64e-4341-99de-a349552e702d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:49.762806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:49.763745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:49.764652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:49.769261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 435/869] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:49.770202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781051b-e64e-4341-99de-a349552e702d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.805777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.809034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:49.831313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 435/870] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:49.855675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.911900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781051b-e64e-4341-99de-a349552e702d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:49.949854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:49.950512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:49.950794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:49.951072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:49.951894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:49.952200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:49.953677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:49.966774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 436/871] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:49.972635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:49.982492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029585-1558-43dd-baf6-3cc9f320b9a1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:50.019523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:50.020068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:50.020348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:50.020630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:50.021362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:50.022742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e15677-9607-4f01-9c43-0d81e3c41c97 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:50.074351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:50.087615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 436/872] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:17:49 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:17:50.088030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e791a-82c2-4106-8eeb-2e15fd649504 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:50.140896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:50.141619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:50.141901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:50.142173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:50.143082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=b4710cdb68364a5696a2e33ad6232650) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:50.143402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:50.145014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e62e4-192b-437d-a678-7d076a4429d3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:50.165360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 437/873] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:49 2022] GET /identity/v3/role_assignments?user.id=b4710cdb68364a5696a2e33ad6232650&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:51.439892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e74678-04c0-4840-ab9f-9e243aa5dd9b None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.440385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e74678-04c0-4840-ab9f-9e243aa5dd9b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.440385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e74678-04c0-4840-ab9f-9e243aa5dd9b None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.441982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 437/874] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:51.448025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.448025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.448025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.466791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:51.468169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:51.543231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 438/875] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:17:51.565250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e74678-04c0-4840-ab9f-9e243aa5dd9b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:51.609611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:51.670916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e74678-04c0-4840-ab9f-9e243aa5dd9b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:51.711218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:51.711755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.712033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.712307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.712940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:51.714341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b037d13f-301c-4eda-8898-bb2d40ad2c85 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:51.723646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:51.724657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:51.725527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 438/876] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:51.731916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:51.812645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:51.825707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb9a4-c14e-4caa-86ef-254a58aaf946 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.825982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb9a4-c14e-4caa-86ef-254a58aaf946 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.828077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb9a4-c14e-4caa-86ef-254a58aaf946 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.828330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 439/877] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:51.834169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.835058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.836069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.840532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785a04-d497-430d-a815-ce4241c1a869 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:51.860416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:51.862362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:51.882105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:51.884005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:51.884284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:51.884552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:51.885545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:51.885849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:51.887487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:51.893869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 439/878] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:17:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:51.905265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:51.965158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 440/879] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:17:51.992693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.028529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.033186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090fae2-00a8-4303-b832-5a043db2dd3a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.082186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.083583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.084090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.084555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.087036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:52.087581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.089771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5067-f39d-4990-ace0-41db1757e3a8 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.097571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 440/880] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:17:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:52.099886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecc8a4-b9e7-4ba7-8be4-e2dca5ed6064 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.147348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.148765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.154899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.154995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.155756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.162998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc924aa8-218f-47dc-ad85-b407aed6c88d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.205630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4cad9-450b-4b7f-b30c-ec7866d5f0e5 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.206011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4cad9-450b-4b7f-b30c-ec7866d5f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.206370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4cad9-450b-4b7f-b30c-ec7866d5f0e5 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.208007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 441/881] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:52.214200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8729-4abb-424c-8aaa-3cd2150e0fd8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.214490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8729-4abb-424c-8aaa-3cd2150e0fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.214752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8729-4abb-424c-8aaa-3cd2150e0fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.230925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.235664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-287d8729-4abb-424c-8aaa-3cd2150e0fd8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:52.236919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-287d8729-4abb-424c-8aaa-3cd2150e0fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:52.256364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 441/882] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:17:51 2022] POST /identity/v3/services => generated 269 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:17:52.329670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac299c-66fa-4ccf-9028-7197120f52a4 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.329964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac299c-66fa-4ccf-9028-7197120f52a4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.330251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac299c-66fa-4ccf-9028-7197120f52a4 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.331941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 442/884] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:52.332073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 442/884] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:17:52.338832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.339122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.339400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.357016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac299c-66fa-4ccf-9028-7197120f52a4 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.359404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:52.360618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:52.443437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 443/885] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:52.468005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.486568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.499720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac299c-66fa-4ccf-9028-7197120f52a4 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.543334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.543800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.544946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.545570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.546046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.546807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.548232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:52.560308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:52.561531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:52.562522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 443/886] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:52.569144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.590071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f386b7-5b97-417b-bcab-890b64bc63bf None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.629938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.630596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.630898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.631144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.642939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:52.648805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:52.650004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:52.650967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 444/887] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:52.655886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.686264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6d18-ae70-48f0-96cd-fd65ad3b5dd1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.734712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.735349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.735642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.735921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.736831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:52.737212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.738865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.746048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 444/888] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:52.750673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.777660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.786532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e076fa-4232-408a-a458-24e48714ffda None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.813493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:52.827688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.828661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.829195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.829713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.830739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:52.831297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.832881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.848335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 445/889] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:52.849776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.868731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffdc65-f680-410d-aa14-a5314a1cadcd None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:52.913784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:52.914513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:52.914806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:52.915086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:52.916085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:52.916304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:52.917921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:52.928270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 445/890] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:52.929420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:52.966759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8125b80-b89a-4932-b768-5181477e4771 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.004904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.005916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.006452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.007840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.019514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.021566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.025680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:53.042037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092ff9-0d00-4008-8253-e1873b7cc8b5 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:53.043128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:53.044019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:53.044938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:53.044938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:53.044938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:53.044938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:53.051916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 446/891] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:53.053036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:53.080323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.080957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.081270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.081552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.082377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] RBAC: Adding query filter params (interface=public, service_id=cf684678038c4c1d875965cf4e43cfbe, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:53.082754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.084201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.090575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 446/892] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:17:52 2022] GET /identity/v3/endpoints?service_id=cf684678038c4c1d875965cf4e43cfbe&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:53.095990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:53.096726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:53.169882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9736-dfd0-464b-aaa0-a2f08f42c4ed None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.207538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.208362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.208749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.209250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.210053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:53.210238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365a14b-09c3-4e87-8b8a-c85b9b54d2f6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.211076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.212751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.231301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 447/893] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:53.232389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:53.248715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.249690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.250128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.250576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.251417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.253184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.265276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 447/894] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:17:53.342575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:53.349320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bbb95-8e1d-43b7-b064-3eefb0b93ee0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.388216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.388878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.389200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.389479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.397686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.399730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:53.413362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:53.415231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:53.416185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 448/895] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:53.423667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:53.443326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:53.537474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da456c6-43a1-4930-ae90-7e2e3de7ec4e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.575990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.576964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.577577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.578076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.579064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:53.579572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.581218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.599795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 448/896] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:53.609040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:53.714577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffc3-230b-4d4e-86f1-7ea4295d3992 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:53.749500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:53.750126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:53.750409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:53.750683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:53.777889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:53.824509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:53.826362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:53.928763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 449/897] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:17:53.938127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.015172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.048533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a2fe-1ef1-4f01-8036-7305e701bb8b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.084848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.086009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.086294 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.086572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.097382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.098843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-ae9a284c-4fd3-43e3-87f1-fac3a3f96d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:54.115159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:54.116183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:54.117737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:54.117737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:54.117737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:54.117737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:54.117737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 449/898] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:54.124039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.185432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.230143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657e73e-1dee-44b6-9021-151e6e3356a5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.263087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3be32e-0a59-404f-8460-222bd3d8fce4 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.263535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3be32e-0a59-404f-8460-222bd3d8fce4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.264010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3be32e-0a59-404f-8460-222bd3d8fce4 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.266045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 450/899] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:54.266775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.267616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.268024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.268421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.269278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:54.269727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.271331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.271446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.271830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.272209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.284723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 450/900] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:54.289968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.292558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:54.294972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:54.376134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 451/901] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:17:54.398702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.409597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e43265-98b1-41e5-8a53-c84192f565d2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.426423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.445852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.446783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.447064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.447345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.454377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.455824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.463263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-7aec21d5-2974-46ac-84ab-792188b2b3ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:54.468783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:54.469909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 451/902] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a97f8-8a1d-4d58-8f52-33d2498de21b None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.478930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a97f8-8a1d-4d58-8f52-33d2498de21b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.479634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a97f8-8a1d-4d58-8f52-33d2498de21b None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.479634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 452/903] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:54.479634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a97f8-8a1d-4d58-8f52-33d2498de21b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.522713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042efd5-cf66-46e0-8716-227c30e2473d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.560473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.561159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.561398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.561679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.562373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.563739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df1d3a-dd44-468b-8f4d-03cc6c412cb3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.590900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a97f8-8a1d-4d58-8f52-33d2498de21b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.625441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.626065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.626394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.626629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.627364 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:54.627674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.629125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.631830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 452/904] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:17:54 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:17:54.633807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.634075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.634349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.655360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:54.656579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:54.661196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 453/905] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:54.662639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.717988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.758732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 453/906] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:54.780117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:54.803074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ca6b8-26a1-4e2f-9ee7-e6b947d2df6a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.833369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:54.846750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.847377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.847658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.847935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.850573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] RBAC: Adding query filter params (role.id=a8e3f51168f449f2a4fbf199e334ae56, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=b4710cdb68364a5696a2e33ad6232650) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:54.850573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.850573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.856729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 454/907] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/role_assignments?user.id=b4710cdb68364a5696a2e33ad6232650&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=a8e3f51168f449f2a4fbf199e334ae56 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:54.905511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f9a8-77c1-4230-94a3-527fbb172bc2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:54.948110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:54.948785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:54.949103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:54.949385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:54.950030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:54.951506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:54.961750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:54.962754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:54.963589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 454/908] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:54.970776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.078998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:55.081826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb2a9a-02bc-414c-a276-b6661a46a851 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.117097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.117729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.118012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.118287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.118977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:55.119282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.120704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.127945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 455/909] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:55.134283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.245201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52590786-7bc2-430c-890f-03a8bd1e82f0 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.254655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:55.280519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.281226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.281506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.281779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.282460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:55.282760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.284130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217480e-94f1-4223-b95a-6cfddb7f61dc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.290387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 455/910] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:55.296300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.402053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ad9-2daa-4fed-97b8-c8b000b7b8a9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.424396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8179-52fe-4001-8faa-9ab275465246 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.424745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8179-52fe-4001-8faa-9ab275465246 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.424967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a8179-52fe-4001-8faa-9ab275465246 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.426677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 456/911] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:17:55.432482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.432775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.433079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.439431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.440237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.440665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.441153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.442069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] RBAC: Adding query filter params (interface=public, service_id=e0bb7d03935e4e239010839fccb648ed, region_id=RegionOne) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:55.442526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.444035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.452406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:55.452874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 456/912] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/endpoints?service_id=e0bb7d03935e4e239010839fccb648ed&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:55.453851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:55.458205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.491266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:55.542087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 457/913] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:55.566354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.576298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff948d4-f533-436d-9eaf-35c5f985b475 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.611948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.612648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.612932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.613238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.613897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.615261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.621900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 457/914] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:17:55.684142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:55.688040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8390ef-6495-48a9-9ce6-914ff57d00b6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.726560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.727316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.728139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.728416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.729089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.730894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:55.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:55.742827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:55.743972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 458/915] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:55.751588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:55.858972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:55.868552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c25c2-0ee1-47ae-be40-9380cead71bb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:55.903301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:55.903928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:55.904215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:55.904552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:55.905282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:55.905593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:55.906962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:55.912287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 458/916] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:55.918727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:56.029159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cc462-9d26-4a67-9df7-0fd355d30982 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:56.066840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:56.067468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.067752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.068032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:56.068722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:56.069053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:56.070483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160792d1-e596-44ef-8890-dc827b6439b1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:56.076065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 459/917] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Jul 29 14:17:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:56.082060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:56.192085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a666c7-7d99-47d8-af1b-0520d455de5e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:56.228116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:56.229185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.229666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.230178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:56.231101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:56.232657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd928c52-a19b-4f35-bc09-bb281e276c30 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:56.317113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 459/918] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:17:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:56.845953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd464b7-9c23-4d01-8c07-7fcb05b33e26 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.846486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd464b7-9c23-4d01-8c07-7fcb05b33e26 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.846486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd464b7-9c23-4d01-8c07-7fcb05b33e26 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:56.848209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 460/919] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:56.854453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.855008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.855008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:56.874298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:56.875496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:56.884611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:56.956887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 460/920] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:56.977074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b979da-d025-40cd-94f0-4de4684afd43 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.977364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b979da-d025-40cd-94f0-4de4684afd43 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.977646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b979da-d025-40cd-94f0-4de4684afd43 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:56.979291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 461/921] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:56.981610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:56.985184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:56.985595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:56.985967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.005513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:57.006800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:57.033030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.093015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 462/922] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:57.108851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.114511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90a215-34bb-4e1b-94ce-9b99053568f7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.159038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.159954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.160435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.160909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.165290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.166935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:57.176840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:57.177868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:17:57.178698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:57.186724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 461/923] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:57.187828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.190961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.232574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8d268-a12c-4168-9591-5941286e9da8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.278288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.279294 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.279801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.280274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.291445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:17:57.296526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:57.297569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:17:57.298405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:57.305969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 463/924] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:57.307077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.307185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d67f8-aede-4ee3-9903-ae01dd60196c None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.327430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.362085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.362919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.363208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.363488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.364573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:57.364891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.366303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.377038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 462/925] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:57.379495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.428264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffa01f-f665-4848-86c2-4587f35c9c1d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.449276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.468315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.468781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.469134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.469417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.470171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:57.470507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.472011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.486085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 464/926] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:57.487500 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.496908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3653-25a0-4deb-883d-5e42e8ac8879 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.543898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.544563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.544859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.545179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.546347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:57.546660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.548725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.562922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 463/927] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:57.563989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.593624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.619118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ef3a2-fe16-4c81-ab7b-2bed25957eba None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.658438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367199-7934-4514-aa92-35d156d8485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.659113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367199-7934-4514-aa92-35d156d8485d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.659397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367199-7934-4514-aa92-35d156d8485d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.659681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367199-7934-4514-aa92-35d156d8485d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.672306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367199-7934-4514-aa92-35d156d8485d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.673801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367199-7934-4514-aa92-35d156d8485d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.678807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e89c-d23c-4472-85d8-d59115b25ea6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-92367199-7934-4514-aa92-35d156d8485d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:57.689825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:57.690741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:57.691623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:57.691623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:57.691623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:57.691623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:57.696144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 465/928] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:57.696974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367199-7934-4514-aa92-35d156d8485d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.714270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.721466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.722457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] RBAC: Adding query filter params (interface=public, service_id=94e9f946e2d742059414509cb954e9e5, region_id=RegionOne) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.736550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 464/929] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/endpoints?service_id=94e9f946e2d742059414509cb954e9e5&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:57.738479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.805256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367199-7934-4514-aa92-35d156d8485d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.838276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:57.844963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.845628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.849513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.849513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.849513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:57.849513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.850256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fe4fe-ab42-47c6-a45e-03197a5beab7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:57.850363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1a9c-1133-4fd2-8a26-aed682d26629 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.866140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 466/930] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:57.868020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17e344-9dfa-4a46-8584-5874ad1a4d80 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.868314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17e344-9dfa-4a46-8584-5874ad1a4d80 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.868595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17e344-9dfa-4a46-8584-5874ad1a4d80 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.870272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 467/931] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:57.870844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17e344-9dfa-4a46-8584-5874ad1a4d80 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:57.889579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:57.890201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.890469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.890735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.891385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:57.892740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb91e9-493d-47ea-83a2-5b4b3bbd5862 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:57.898824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 465/932] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:17:57.901126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:57.901126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:57.901126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:57.920454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:57.921757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:57.982752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.004317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17e344-9dfa-4a46-8584-5874ad1a4d80 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.005438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 466/933] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:58.021569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.056345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.057807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.058832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.059855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.070997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.073802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:17:58.092609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:58.093664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:17:58.094609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:58.101788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 468/934] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:17:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:58.104371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.157328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da88f7-6f80-4458-9ddb-de57a7543ba7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.173508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.203009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.203670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.203956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.204232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.205338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.206724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:58.221162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:58.222156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:17:58.222982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 467/935] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:58.233872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b240ab8-aa89-46e2-bd73-8681757a621f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.234501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.272716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.273605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.273996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.274372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.275390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:58.275798 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.277576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.302472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 469/936] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:58.306960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.308686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.363344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd3a10-ea2d-47e6-9a08-efdbb2ed3355 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.399873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.400498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.400774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.401126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.401843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:58.402144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.403601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.414400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 468/937] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:17:58.415561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.426132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f35ce-b1f0-4b9c-9976-49e1e3e7d198 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.437275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.463768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.464434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.464717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.465020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.477997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.479450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-3b082b7e-f55b-4c56-bfba-6b19fd011e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:17:58.494797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:58.495802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:58.496707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:17:58.496707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:17:58.496707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:17:58.496707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:58.496707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 470/938] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:58.497593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79f6f-8aed-4836-89dd-b9635211ab89 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.497593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79f6f-8aed-4836-89dd-b9635211ab89 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.497744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79f6f-8aed-4836-89dd-b9635211ab89 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.499096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 471/939] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:17:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:17:58.502842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb79f6f-8aed-4836-89dd-b9635211ab89 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.534824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d24a0-d4ec-43fb-a0fb-e3264c43ab0e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.553415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.571584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.572203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.572507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.572756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.573468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:58.573805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.575123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bd6a3-e5b1-403a-a4d3-b19b83ecde92 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.581035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 469/940] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:58.581681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.582449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.582723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.602504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:17:58.603696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:17:58.625113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb79f6f-8aed-4836-89dd-b9635211ab89 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.661209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.661832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.662113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.662389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.663079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:58.663378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.665159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.681491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 470/941] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:17:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:17:58.681491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.690374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 472/942] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:58.691650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.715285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.808878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612ca19-2b78-4677-8018-02e036cf925b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.816739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70b1d5-3425-4430-8737-a1d64c501084 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:58.821353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:58.844475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.845246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.845531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.845810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.846514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.847885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.857153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:58.858379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:58.858819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:58.859096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:58.866265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:58.868447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:58.880423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:58.881544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 473/943] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:58.886332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.957758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 471/944] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:17:58 2022] POST /identity/v3/endpoints => generated 347 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:17:58.959303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:58.981405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:59.007298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982174e-2bc0-41e1-b22a-75daea7718f6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:59.058581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:59.059254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:59.059535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:59.060076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:59.060888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:59.062524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:59.065313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING keystone.server.flask.common [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 29 14:17:59.108924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e116a5-14c9-4814-a374-c4b7e03d00a5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:59.112193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.sql.core [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 14:17:59.112193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 14:17:59.112193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [parameters: {'id': '7b57a80b618c4416bcb16fc52fa4364a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 14:17:59.112193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80331) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b57a80b618c4416bcb16fc52fa4364a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b57a80b618c4416bcb16fc52fa4364a. Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:17:59.129279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result.read() Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 29 14:17:59.134237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application yield resource Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise self.value Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application session.commit() Jul 29 14:17:59.135174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.session.flush() Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application compat.raise_( Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 14:17:59.136058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 14:17:59.136952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result.read() Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:17:59.138348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [parameters: {'id': '7b57a80b618c4416bcb16fc52fa4364a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:59.139199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:59.139964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b57a80b618c4416bcb16fc52fa4364a. Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.140871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 474/945] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Jul 29 14:17:58 2022] POST /identity/v3/projects => generated 262 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:17:59.146490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:59.160289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:59.160978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:59.161302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:59.165068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:59.165068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:59.165068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:59.165068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:59.180943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 472/946] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:17:59.187046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:59.224118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:59.281624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55d519-f090-48e0-9c37-44c1edf5d23a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:59.315797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:17:59.318560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463225-639b-4c32-bf4d-0d1d6832822d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:59.321927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:59.322750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:59.323124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:59.323564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:59.331193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:59.333119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:17:59.346032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:17:59.346967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:17:59.347886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:17:59.352756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 475/947] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:17:59.353658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:17:59.364549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:59.365251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:59.365909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:59.366194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/b4710cdb68364a5696a2e33ad6232650/roles/a8e3f51168f449f2a4fbf199e334ae56` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:59.369705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=b4710cdb68364a5696a2e33ad6232650, role_id=a8e3f51168f449f2a4fbf199e334ae56)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:59.371317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e5f3-bbb6-4d0a-9333-6f9516d79429 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:59.445522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 473/948] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:17:59 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/b4710cdb68364a5696a2e33ad6232650/roles/a8e3f51168f449f2a4fbf199e334ae56 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:17:59.470051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367124b-a938-4dca-b2d1-3e60efcdd751 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:17:59.509578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:17:59.510132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:17:59.510411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:17:59.510685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:17:59.511431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:17:59.511735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:17:59.513240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5cad4-9f38-400f-b08f-84281b788c2b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:17:59.536101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 476/949] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:17:59 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:00.027518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708e1a-6a59-4d6e-9bb2-0e0ddb7ba8dc None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.028159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708e1a-6a59-4d6e-9bb2-0e0ddb7ba8dc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.028159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708e1a-6a59-4d6e-9bb2-0e0ddb7ba8dc None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.029754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 474/950] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:00.035370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.035824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.036026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.055788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:00.057343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:00.109849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:00.131471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 477/951] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:00.155601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708e1a-6a59-4d6e-9bb2-0e0ddb7ba8dc None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:00.253700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:00.270142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708e1a-6a59-4d6e-9bb2-0e0ddb7ba8dc None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:00.313289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:00.313289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.313289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.313289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.313705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:00.313705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:00.322857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:00.324120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:00.325063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:00.327547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 475/952] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:18:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:00.334275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:00.441147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8501efd-3cd5-43e2-8207-b10d3b31def4 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:00.476620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:00.477262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.477541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.477814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.478494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:00.478794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:00.480336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:00.485627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 478/953] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:00.491129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:00.598427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85277e4d-571f-4324-8ce8-48478c2af3ec None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:00.634582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:00.635527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.635991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.636548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.637431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:00.637928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:00.639478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512347-4343-4af7-ac35-1950583df3de None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:00.645276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 476/954] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:00.652127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:00.704445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:00.762188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd06b2-70b8-4805-9a21-6b50a840dac9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:00.799061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:00.799685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:00.799965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:00.800242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:00.800961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:00.802349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e41335-881d-462f-8437-8731156e0f2c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:00.897675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 479/955] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:18:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:01.150653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b395738-ef7d-4b52-96ad-fd4a74b15f0a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.151171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b395738-ef7d-4b52-96ad-fd4a74b15f0a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.151171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b395738-ef7d-4b52-96ad-fd4a74b15f0a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.152881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 477/956] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:01.158433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.158883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.158971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.178046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:01.179386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:01.259431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 480/957] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:01.281788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b395738-ef7d-4b52-96ad-fd4a74b15f0a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:01.298298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:01.389145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b395738-ef7d-4b52-96ad-fd4a74b15f0a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:01.423412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:01.423968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.424245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.424515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.425163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:01.426519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-8c29d35e-34b0-4680-87b8-449afdde1e26 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:01.436224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:01.437381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:01.438633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 478/958] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:01.443547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:01.473907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:01.554984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a443f7-bed3-457a-b90f-37f3c64fbd2a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:01.590879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:01.591511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.591790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.592069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.592758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:01.593093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:01.594484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:01.599777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa2718-3a06-4ba6-942b-27e7b30c0539 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.600008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 481/959] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:01.600164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa2718-3a06-4ba6-942b-27e7b30c0539 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.600868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa2718-3a06-4ba6-942b-27e7b30c0539 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.602497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 479/960] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:01.605544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:01.607134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.607416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.607844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.626310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:01.627497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:01.698940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 480/961] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:01.708751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:01.718168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b6eb-c77d-40b2-93fb-0efb1c185abf None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:01.721661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:01.732897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:01.762575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:01.763559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.764373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.764655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.765561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:01.765670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:01.767516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfdd5-ab3c-4900-adf0-c4813792ca66 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:01.771995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 482/962] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:01 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:01.773240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1d2c-875c-4fd5-999c-dedea23402e4 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.773460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1d2c-875c-4fd5-999c-dedea23402e4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.773794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b1d2c-875c-4fd5-999c-dedea23402e4 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.779681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 483/963] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:01.781628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.781628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.781864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.801660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:01.802892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:01.846434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4560cea-f987-4a8c-bc7e-fbd82b939a5e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:01.885375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:01.886069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:01.886351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:01.886630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:01.890329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 484/964] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:01.912343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:01.918350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:01.918657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:01.919611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:01.920460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 481/965] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:18:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:01.925805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.017356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.025400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.045514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0ce78-d9d0-4f43-83bc-c03919a0952e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.065466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0c9a-bd17-4be7-87c6-2be3f3093049 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.084803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.086486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.086912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.087348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.088232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.089901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.109510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.110279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.110634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.111040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.111826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:02.112199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.113682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.125046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 482/966] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:18:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:02.128829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.141709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.sql.core [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 14:18:02.141709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 14:18:02.141709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [parameters: {'user_id': 'e1bf3449d91d41f0a42982e1eec862eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 14:18:02.141709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80331) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result.read() Jul 29 14:18:02.160231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:18:02.161595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application yield resource Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application next(self.gen) Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.session.rollback() Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.force_reraise() Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise self.value Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application session.commit() Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 29 14:18:02.162594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._prepare_impl() Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.session.flush() Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._flush(objects) Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application compat.raise_( Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application flush_context.execute() Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rec.execute(self) Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 29 14:18:02.165110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 29 14:18:02.166521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result = self._query(query) Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application conn.query(q) Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application result.read() Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 29 14:18:02.168637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e1bf3449d91d41f0a42982e1eec862eb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:02.171877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 29 14:18:02.173936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 485/967] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Jul 29 14:18:01 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:02.177070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.268156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad411e91-1419-414c-a4c4-657545a5ef61 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.297752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.303682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.304861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.305371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.305649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.306015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadff487-d4dc-48dc-abe6-e0a435b8b64e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.317032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.318552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:02.334621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:02.335651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:02.336572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:02.336572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:02.336572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:02.336572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:02.340800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 483/968] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:02.341691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.348136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.349140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.349732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.350218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.358971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.360986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:18:02.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:02.375685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 29 14:18:02.376643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:02.379336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 486/969] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:02.389680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.409487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.462636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fae0-4c81-44f9-a127-68d94091dd85 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.510174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.511374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.511924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.517268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.518393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:02.519315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.521200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.522025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a034e-30cf-4280-8ca1-497e5d72c952 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.537456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 484/970] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:02.544427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.564748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.565403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.565679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.565949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.566714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:02.567016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.568432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.599603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.613033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 487/971] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:02.670746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b862bcc-698f-4e91-988e-f0a8ec9c1ead None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.708772 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.709464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.709743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.710015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.718215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.719683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:18:02.733360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:02.734411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:02.735389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 485/972] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:02.746146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.822326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:02.858186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee78d06-b30e-4ce2-b997-677f2067ee14 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:02.895187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:02.895825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.900198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.900198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.900198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:02.900198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:02.900198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e8158-d410-4ab1-bd59-cd8e7f69ac4f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:02.920874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 488/973] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:02.926148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:02.980661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091009f2-52d0-483c-8080-146521e65008 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.980958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091009f2-52d0-483c-8080-146521e65008 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.981264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091009f2-52d0-483c-8080-146521e65008 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:02.982623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 489/974] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:02.987523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:02.987805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:02.988077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.009194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:03.010757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:03.038943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c591e-6afd-46f9-8f8f-8abbe2434f96 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.040492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.076132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.077151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.077581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.078009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.089991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 490/975] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:03.112498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.151729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.153332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.180540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.228754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487f91-9810-4c99-8ab9-045ccad6b1a0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.261066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 486/976] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:02 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 14:18:03.266598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.268457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.272417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.273797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.274093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.274449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.275243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.276468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:03.286654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:03.287701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:03.289192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 491/977] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:03.294647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.369792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ba150-e201-4b32-b7cc-6557d1379e72 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.403963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.404168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a1aeb-05ef-4f72-8ff1-6c76f5e6539d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.404663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.404944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.405280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.416357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.417829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.422862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:03.433867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:03.439974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:03.440884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:03.440884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:03.440884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:03.440884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:03.440884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 487/978] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:03.442408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.444296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.445086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.445374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.445652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.446468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:03.446830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.448206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.458362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 492/979] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:03.459267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.509525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.558300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad0968-1b0c-4adc-b183-30bc83313bc7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.573229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6bd1-9782-4eea-92a0-490a5c2782aa None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.594003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.594617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.594898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.595171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.595861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:03.596162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.597829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.608948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396feb1-1630-4291-981d-06231d37171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.609603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396feb1-1630-4291-981d-06231d37171c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.609885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396feb1-1630-4291-981d-06231d37171c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.610331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 488/980] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:03.613234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396feb1-1630-4291-981d-06231d37171c None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.614141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396feb1-1630-4291-981d-06231d37171c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:03.614609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396feb1-1630-4291-981d-06231d37171c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.615510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.615999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396feb1-1630-4291-981d-06231d37171c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.627915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 493/981] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:03.629204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396feb1-1630-4291-981d-06231d37171c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.680812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.724933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78434b64-825b-4451-82b3-3a18b976f080 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.740980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.741465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396feb1-1630-4291-981d-06231d37171c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.762209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.762855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.763135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.763404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.770969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.772403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.779299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.779919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.780199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.780475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.781261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] RBAC: Adding query filter params (interface=admin, service_id=94e9f946e2d742059414509cb954e9e5, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:03.781574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.782927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:18:03.785145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:03.786034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:03.787822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 489/982] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:03.796030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 494/983] 10.222.0.49 () {56 vars in 1249 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/endpoints?service_id=94e9f946e2d742059414509cb954e9e5&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:03.796193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.796665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.899171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578de8-6555-4f4e-bd67-7df60e8cd2e3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.904034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ca2c-8999-4587-9dfe-427dd85ec66a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:03.915946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:03.933182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.933800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.934079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.934352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.935354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:03.935661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.937102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778ad42-cdd3-408e-abdd-5f7126b4b20a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.939849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:03.940451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.940729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.941009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.941664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:03.942996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:03.949988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 490/984] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:03.955333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 495/985] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:18:03.955470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cc2e7-28db-4ccf-b823-edf43f0c3bdc None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.955470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cc2e7-28db-4ccf-b823-edf43f0c3bdc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.955470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cc2e7-28db-4ccf-b823-edf43f0c3bdc None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.955470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 491/986] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:03.956303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:03.958358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:03.958647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:03.958923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:03.980695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:03.981962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:04.014064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:04.061676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 492/987] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:04.080069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fcf53-0427-4526-998a-6c90a0843d85 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:04.085313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:04.119817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:04.120436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:04.120716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:04.121020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:04.121793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] RBAC: Adding query filter params (role.id=a8e3f51168f449f2a4fbf199e334ae56, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=b4710cdb68364a5696a2e33ad6232650) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:04.122301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:04.123515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16386e6e-02ec-4802-bb64-ff0aea9f121f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:04.130767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 496/988] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:03 2022] GET /identity/v3/role_assignments?user.id=b4710cdb68364a5696a2e33ad6232650&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=a8e3f51168f449f2a4fbf199e334ae56 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:04.212778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ff845-a895-4e29-8a15-cf9d4577693d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:04.250504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:04.251252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:04.251609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:04.251958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:04.252817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:04.254334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd951-9806-458b-a95d-1e344e40ca5c None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:04.334807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 493/989] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:18:04 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:05.962023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf25c36-e7bf-4507-9769-3c39c5a98f2e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:05.962555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf25c36-e7bf-4507-9769-3c39c5a98f2e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:05.962555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf25c36-e7bf-4507-9769-3c39c5a98f2e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:05.964055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 497/990] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:05.969059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690f1f-d5d3-4f62-abb8-0fc4c9663342 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:05.969481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690f1f-d5d3-4f62-abb8-0fc4c9663342 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:05.969666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690f1f-d5d3-4f62-abb8-0fc4c9663342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:05.987919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e8690f1f-d5d3-4f62-abb8-0fc4c9663342 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:05.989110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e8690f1f-d5d3-4f62-abb8-0fc4c9663342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:06.006094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.062900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 494/991] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:06.084652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf25c36-e7bf-4507-9769-3c39c5a98f2e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.127418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.194711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9e9d-406c-498e-91f7-d3f4728e47f4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.195596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9e9d-406c-498e-91f7-d3f4728e47f4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.195862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9e9d-406c-498e-91f7-d3f4728e47f4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.197267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 495/992] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:06.202251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.202465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf25c36-e7bf-4507-9769-3c39c5a98f2e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.202590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.202801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.220722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:06.221860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:06.242470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.243117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.243412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.243676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.244314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.245722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:06.254635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:06.255624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:06.256479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 498/993] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:06.261652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.299037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 496/994] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:06.321492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.369048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b020d-17fa-4b3e-8359-561bd558bdef None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.370636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.404827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.405477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.405755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.406027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.406697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:06.406997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.408042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.408363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b206f1-ac8d-4dc5-87d8-6e47f61eac48 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.418511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 499/995] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:06.418511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dbc30-c56f-4df6-b288-bdd312f6b003 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.418511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dbc30-c56f-4df6-b288-bdd312f6b003 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.418511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dbc30-c56f-4df6-b288-bdd312f6b003 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.418511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 500/996] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:06.421138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dbc30-c56f-4df6-b288-bdd312f6b003 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.428631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776937f-ff6b-448e-96be-54ead74de78f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.465001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.465660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.465935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.466203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] PATH_INFO: `/v3/services/image` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.466832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.468178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-f03cbf75-97ed-4a94-a3ec-ffde9c6dfe49 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:06.477679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:06.478936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.479882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 497/997] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:06.480681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.480681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.480681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.498454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:06.499635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:06.527448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dbc30-c56f-4df6-b288-bdd312f6b003 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.564853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.565536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.565817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.566090 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.566537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 498/998] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:06.566760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:06.567053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.567375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.568413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.573680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 501/999] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:06.579110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.611028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.646013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.670334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0fef-d724-436a-9ab6-641b0a411d8e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.682566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418c6-a51f-4290-82ed-0600eb4c8424 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.705208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.705825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.706105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.706377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.707050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] RBAC: Adding query filter params (name=image) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:06.707353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.708715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.713489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 499/1000] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:06.714256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.720480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.721124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.721404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.721682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.722417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.723795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.796582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 502/1001] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:18:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:06.797435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.814794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bce0c-5722-4560-b266-341dcaf5d920 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.866247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.868430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.868898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.869351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.870198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.871790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.874126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:06.883482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:06.884893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:06.885951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:06.888289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 500/1002] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:06.895455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:06.909392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:06.920961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aa194-b6f9-48dd-964a-c74c48d04d3c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:06.960356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:06.961569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:06.961857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:06.962138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:06.963175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] RBAC: Adding query filter params (type=image) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:06.963535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:06.965142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:06.970702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 503/1003] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:07.021240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5019e06-6158-4639-8110-6e042fd07553 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:07.061542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:07.062085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:07.062366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:07.062644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:07.063341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:07.063648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:07.065267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:07.071804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 501/1004] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:07.078091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:07.198072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209895c-61fe-4795-8fc2-ea758cb23662 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:07.233625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:07.234622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:07.235055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:07.235509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:07.236346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:07.236900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:07.238428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:07.244867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 504/1005] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:07.250847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:07.316123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:07.356426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985cc0-fe17-4853-b4e3-90e97a359360 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:07.391399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:07.392026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:07.392308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:07.392585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:07.393379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] RBAC: Adding query filter params (interface=public, service_id=2b964ab3191949879f38e157a3a47f37, region_id=RegionOne) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:07.393694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:07.395068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea29563-170a-4dd7-bb77-ab439b4c46c6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:07.404746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 502/1006] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:18:07 2022] GET /identity/v3/endpoints?service_id=2b964ab3191949879f38e157a3a47f37&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:07.406065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:07.426915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:07.514248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5027-580c-4a77-93a2-c157539d7f43 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:07.549305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:07.549928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:07.550212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:07.550492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:07.551144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:07.552513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63705e6-f5ca-4906-9494-492f56638f97 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:07.561181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 505/1007] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:18:07 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:18:08.846780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991444e9-4cc7-4fab-b06f-d3e99d54dd0b None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:08.847306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991444e9-4cc7-4fab-b06f-d3e99d54dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:08.847383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991444e9-4cc7-4fab-b06f-d3e99d54dd0b None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:08.848853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 503/1008] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:08.853947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915789-6b63-46d1-85e8-2c04ff57e610 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:08.854372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915789-6b63-46d1-85e8-2c04ff57e610 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:08.854563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915789-6b63-46d1-85e8-2c04ff57e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:08.872870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3d915789-6b63-46d1-85e8-2c04ff57e610 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:08.874077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3d915789-6b63-46d1-85e8-2c04ff57e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:08.942702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 506/1009] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:08 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:08.965702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991444e9-4cc7-4fab-b06f-d3e99d54dd0b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:08.998968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f7ba-8e7e-44e3-9798-40faec5a8d42 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:08.999273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f7ba-8e7e-44e3-9798-40faec5a8d42 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:08.999532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f7ba-8e7e-44e3-9798-40faec5a8d42 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.001181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 507/1010] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:09.005702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.005983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.006256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.024554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:09.025759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:09.033217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:09.072803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991444e9-4cc7-4fab-b06f-d3e99d54dd0b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:09.096246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 508/1011] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:09.105805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:09.110052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:09.110696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.110974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.111245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.112388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:09.113841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:09.120407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 504/1012] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:09.120539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:09.122185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:09.232634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c779d-9d04-4a58-8fdf-6c1a7a61ac11 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:09.239307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d72a-3e03-48a1-993f-4558a7c526a7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:09.269550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:09.270289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.270573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.270921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.271546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:09.272910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097db0-d30e-4cfc-a0d8-ae8905ce18e3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:09.274346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:09.274968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.275248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.275522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.276211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:09.277660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e37f-93a4-45da-9f7a-eb9422907f69 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:09.369659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 509/1013] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:18:09 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:09.405055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 505/1014] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:18:09 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:09.590739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5a81d-5e8a-4ad0-92f6-e0bef4203435 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.591025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5a81d-5e8a-4ad0-92f6-e0bef4203435 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.591306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5a81d-5e8a-4ad0-92f6-e0bef4203435 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.592673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 510/1015] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:09.598113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.598403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.598678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.617673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:09.618862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:09.664360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:09.696480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 506/1016] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:09 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:09.719194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5a81d-5e8a-4ad0-92f6-e0bef4203435 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:09.722181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:09.827445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5a81d-5e8a-4ad0-92f6-e0bef4203435 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:09.864289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:09.865073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:09.865263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:09.865532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:09.866168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:09.868058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:09.877462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:09.878478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:09.879337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 511/1017] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:09.885437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:09.989774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e590-ebbf-41de-a492-73e5defa6a09 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:10.023722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:10.024352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:10.024633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:10.024905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:10.025621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:10.025928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:10.027305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:10.036739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 507/1018] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:10.038278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:10.109443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:10.143540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343ef8f-2e2a-47b1-8274-b837f8bd94c0 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:10.186220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:10.186844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:10.187123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:10.187396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:10.188069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:10.188372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:10.189765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2d44b-e563-4165-b1e9-9d8ace3f5863 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:10.195141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 512/1019] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Jul 29 14:18:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:10.200814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:10.236403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:10.302663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b9fc-cbc6-4c13-bd5c-59ca35289727 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:10.337405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:10.338015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:10.338296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:10.338566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:10.339275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:10.340631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b5508-2450-4f6b-b6a9-d4c0f18a5af7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:10.417033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 508/1020] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:18:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:11.432629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303dc5-7885-443a-9a52-265d3aff746b None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.432629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303dc5-7885-443a-9a52-265d3aff746b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.432629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303dc5-7885-443a-9a52-265d3aff746b None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.439763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.439763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.439763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.445497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 513/1021] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:11.472906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:11.474454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:11.485059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820acf2-64a0-4a43-a5ca-fbe1cd4201a3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.485373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820acf2-64a0-4a43-a5ca-fbe1cd4201a3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.485655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820acf2-64a0-4a43-a5ca-fbe1cd4201a3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.490195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 514/1022] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:11.493238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.493528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.493807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.514228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:11.515456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:11.569032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 509/1023] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:11.588143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:11.603262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 515/1024] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:11.609706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:11.630222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:11.698803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:11.719796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eb505-3a8f-496e-ad30-144715ad0621 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:11.740570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad9d2b-2a98-414c-ae3a-6f3b9bf5d8e8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:11.756961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:11.758711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.759315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.759759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] PATH_INFO: `/v3/services/image` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.760537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:11.762200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:11.771958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:11.773563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:11.775180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 510/1025] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 14:18:11 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:11.779670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:11.782226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:11.782724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.783005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.783277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.784471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:11.785916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:11.787513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 516/1026] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:11.793228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:11.871155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:11.891837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9319c2-45c5-48a8-bde3-c22228c43a55 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:11.895202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4226ae-98a4-439b-ba1a-16458396a604 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:11.921699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:11.928520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:11.929806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.930091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.930222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:11.930545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.930822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:11.931097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:11.931222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:11.931365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:11.931532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:11.932048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:11.932911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:11.933533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:11.942768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 511/1027] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:11.944235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:12.027870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 517/1028] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:18:11 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:12.047302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f441-ecb0-4d26-8203-3744c861a5e6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:12.087362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:12.088030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.088311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.088587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.089318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] RBAC: Adding query filter params (type=image) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:12.089629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:12.091028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:12.101848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 512/1029] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:12.104020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:12.220673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1779-a8de-456f-8c84-002c974a2d7c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:12.256678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:12.257377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.257660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.257935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.258688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] RBAC: Adding query filter params (interface=public, service_id=4a86fecfc43341c5b3391b0d05f9cfa8, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:12.258998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:12.260389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d1adb-2e27-435e-ae96-6b0eac2c3788 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:12.266906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 518/1030] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:18:12 2022] GET /identity/v3/endpoints?service_id=4a86fecfc43341c5b3391b0d05f9cfa8&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:12.271563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:12.286332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:12.376755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5ac5a-25e4-4b0a-b8c4-c23d432a2921 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:12.410221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:12.410840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.411117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.411389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.412038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:12.413444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acc3eb-1bbd-42e4-9c73-375a38d1a8e3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:12.420953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 513/1031] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:18:12 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:18:12.547421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c2bf-cb0e-4f1c-aa88-bf754d0a39a1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.547927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c2bf-cb0e-4f1c-aa88-bf754d0a39a1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.547927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c2bf-cb0e-4f1c-aa88-bf754d0a39a1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.549487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 519/1032] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:12.555485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.555999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.556303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.576202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:12.577428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:12.645892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 514/1033] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:12.669759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c2bf-cb0e-4f1c-aa88-bf754d0a39a1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:12.721189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:12.776005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c2bf-cb0e-4f1c-aa88-bf754d0a39a1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:12.812237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:12.812797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.813107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.813386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.824819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:12.829773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:12.830783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:12.831640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:12.833129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 520/1034] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:18:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:12.837133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:12.840295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:12.940829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c96b-a704-4a68-a09e-aeac74616095 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:12.975884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:12.977170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:12.977453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:12.977726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:12.978448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:12.978753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:12.980113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:12.988471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 515/1035] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:18:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:12.994733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:13.096763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847cb1-abf1-4bd3-8ba2-5380a7b064c3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:13.106371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:13.132247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:13.133622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:13.133906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:13.134179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:13.146108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:13.147606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:13.169723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:13.170689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:13.171583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:13.171583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:13.171583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:13.171583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:13.177062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 521/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:13.177199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:13.250125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:13.286460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1077a2-b5d6-4896-9441-ce2650e8bea1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:13.321382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:13.321989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:13.322264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:13.322535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:13.323213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:13.323511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:13.324863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:13.337536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 516/1037] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:13.342888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:13.442300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74359-8353-4cb7-b015-2a24d8fe48d7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:13.475412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:13.476079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:13.476934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:13.477258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:13.484941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:13.486407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:18:13.497464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:13.498419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:13.499806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:13.501147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 522/1038] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:18:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:13.504168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:13.607265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010aced-34d3-45dc-8c96-b19212a101af None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:13.633600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:13.642424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:13.643038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:13.643314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:13.643582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:13.644438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:13.644851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:13.646347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:13.668025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 517/1039] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:18:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:13.669667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:13.692883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:13.772593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe3776-12e5-4aef-ab49-3deebc66d00c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:13.807014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:13.807634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:13.807909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:13.808179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:13.876181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:13.877677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab448c6d-1878-4957-b6da-26acefd5c91d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:13.898649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:13.983745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 523/1040] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:13 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 14:18:13.984828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.086098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26915-187b-412b-895a-59eb3e7d56ce None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.117395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67ad2-4c84-4aa9-86e8-c0a2b193fa11 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.117694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67ad2-4c84-4aa9-86e8-c0a2b193fa11 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.117970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67ad2-4c84-4aa9-86e8-c0a2b193fa11 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.119467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 524/1041] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:14.120305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.120942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.121245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.121519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.124130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.124415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.124688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.132497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.133960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.142656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:14.143894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:14.155836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:14.156823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 518/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:14.157802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.198758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.226778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 525/1043] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:14.250082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.268786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e30fa7-8b16-4d6a-86d2-65b92559acce None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.300799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.309567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.310441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.310771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.311086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.311865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:14.312168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.313562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.326005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 519/1044] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:14.331315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.356145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a0d6c-ce13-4566-89ec-007c1af5e2fe None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.392691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.393363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.393639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.393903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.405250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:14.411213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:14.412363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:14.413273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 526/1045] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:14.418432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.430932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.437086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe6a9e-44e0-415c-a100-fa9e8f4724c5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.472309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.472958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.473270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.473541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.480829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.482297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6890ae02-296e-45e9-b276-48ce93862dd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:18:14.493959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:14.494860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:14.495766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 520/1046] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:14.496576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576df2-0f0b-4ac5-8d0a-f798b2cfd078 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.496576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576df2-0f0b-4ac5-8d0a-f798b2cfd078 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.496856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576df2-0f0b-4ac5-8d0a-f798b2cfd078 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.498118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 521/1047] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:14.500706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf576df2-0f0b-4ac5-8d0a-f798b2cfd078 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.526360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90731c-91bb-41ad-9a4b-a99a7f8951a4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.537060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.560906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.561560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.561840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.562110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.562820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:14.563123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.564476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a201b-6da4-432b-9b77-4e8076546aa5 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.573255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 527/1048] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:14.574517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.574805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.575164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.593330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:14.594695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:14.603789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf576df2-0f0b-4ac5-8d0a-f798b2cfd078 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.637936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.638559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.638798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.639125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.639877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:14.640182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.641644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.654139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 522/1049] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:14.655178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.667852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 528/1050] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:14.668802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.675708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.758161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c177-fdcb-48a8-a011-2898b8831caf None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.770301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84af21-b100-4e93-b32f-b8e583ccbd0d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.771820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.792494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.793160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.793440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.793715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] PATH_INFO: `/v3/users/cdd18741d6e74152803a4dd85a3bda17` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.804200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.805483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.805757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.806032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.806810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] RBAC: Adding query filter params (role.id=a8e3f51168f449f2a4fbf199e334ae56, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:14.807113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.807262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd18741d6e74152803a4dd85a3bda17)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.808530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.808690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.810630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 523/1051] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/users/cdd18741d6e74152803a4dd85a3bda17 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:14.811367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.815277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 529/1052] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=a8e3f51168f449f2a4fbf199e334ae56 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:14.816374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:14.904417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:14.933628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbb65c-c48f-4870-b707-4885ffbf26f3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.950433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c328-bc30-4082-8e72-26a1238c41f1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:14.976285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.977227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.977507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.977779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.978408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:14.979774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:14.988092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:14.988711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:14.989006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:14.989288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:14.997646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:14.998658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:14.999504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 524/1053] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:15.004766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:15.029152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:15.068414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.070346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.121608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f489e8-7ef2-429a-a8ea-62a8060079f9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:15.162745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:15.163305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:15.163657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:15.164628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:15.165299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] RBAC: Adding query filter params (name=image) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:15.165601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.166953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.172044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 525/1054] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:15.177959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:15.180538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:15.185207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 530/1055] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:15.188545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:15.259889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:15.282693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e36dc9-6f97-4d37-8501-e9dc90ede516 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:15.298905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1ded-268c-4cc0-a0f2-83d84429a3f1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:15.320508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:15.321246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:15.321526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:15.321797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:15.322480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] RBAC: Adding query filter params (type=image) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:15.322783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.324601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.329711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 526/1056] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:15.335307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:15.345532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:15.346177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:15.346459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:15.346739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:15.349194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.350628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.352485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 531/1057] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:15 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:15.358547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:15.409174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:15.449202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca3a74-9f41-4f13-b6cd-e10701471156 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:15.463078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0046f-cfae-48d9-9788-6820d52ac449 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:15.488433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:15.489108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:15.489389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:15.489663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:15.490426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.492451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2477f6-aef3-4353-89fe-30685b9ea8dd None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.500026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:15.500652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:15.500937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:15.501253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:15.502036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=96c045ea68c643bc95334d2466df4ba2, user.id=cdd18741d6e74152803a4dd85a3bda17) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:15.502348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:15.503786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521893f-49d5-4886-ab99-f6f4b5d86606 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:15.505660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:15.521036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 532/1058] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:15 2022] GET /identity/v3/role_assignments?user.id=cdd18741d6e74152803a4dd85a3bda17&scope.project.id=96c045ea68c643bc95334d2466df4ba2&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:15.566419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 527/1059] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:18:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:16.831206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eef61-fde0-4348-8b4d-1e60cd517b23 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:16.831809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eef61-fde0-4348-8b4d-1e60cd517b23 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:16.831809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eef61-fde0-4348-8b4d-1e60cd517b23 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:16.833192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 533/1060] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:16.838451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:16.838874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:16.839018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:16.857617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:16.858793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:16.935415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 528/1061] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:16.957436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9eef61-fde0-4348-8b4d-1e60cd517b23 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.061606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9eef61-fde0-4348-8b4d-1e60cd517b23 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.096956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.097557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.097838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.109128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:17.113990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:17.114920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:17.115781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:17.117167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 534/1062] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:18:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:17.121310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.180639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.233129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c183-6739-4e4b-bc9c-dec3cfaf9af8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.268982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.269629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.269912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.270190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.270903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:17.271208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:17.272581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:17.286089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 529/1063] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:17.287036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.347364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.392201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81daaa6-e456-488f-af1c-6ca1e17e04fa None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.425929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.426581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.426862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.427136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.438695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:17.440157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-96b636c5-d2de-4c98-9d72-a6ef49aa6a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:17.454983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:17.455915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:17.456820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:17.456820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:17.456820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:17.456820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:17.456820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 535/1064] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:17.463209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.503097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcd25e-d8cb-40a4-9df2-ce9fb514a659 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.503423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcd25e-d8cb-40a4-9df2-ce9fb514a659 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.503648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcd25e-d8cb-40a4-9df2-ce9fb514a659 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.505168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 536/1065] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:17.510371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade494bc-d27a-404f-b480-2292c6bd91fd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.510655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade494bc-d27a-404f-b480-2292c6bd91fd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.510932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade494bc-d27a-404f-b480-2292c6bd91fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.529504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ade494bc-d27a-404f-b480-2292c6bd91fd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:17.530968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ade494bc-d27a-404f-b480-2292c6bd91fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:17.572080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62600994-3b3f-4ece-a0a2-509934769042 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.583867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.603837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 537/1066] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:17.609630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427177f1-0c05-41b5-8cb6-278081288b6d None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.609926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427177f1-0c05-41b5-8cb6-278081288b6d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.610202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427177f1-0c05-41b5-8cb6-278081288b6d None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.610784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.611407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.611714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 538/1067] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:17.611825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.612012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.612777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:17.613118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:17.614487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:17.616331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.616616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.616924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.628946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 530/1068] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:17.630096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.635690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:17.636885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:17.672142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.704407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 539/1069] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:17.705259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.740910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cd273-cfbf-4053-85ab-3ed2b83cb950 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.779999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.779999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.779999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.779999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.787612 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:17.792588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:17.793619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:17.794494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 531/1070] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:17.795222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.809231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf929b0-d8d7-4555-a90d-19fe360a73a3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.826738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.843952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.844627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.844903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.845203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.852354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:17.853812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:18:17.865514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:17.867201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 540/1071] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:17.868705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:17.898249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:17.902328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4975ce-2014-455b-99b2-e9a0ea592b08 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:17.937240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:17.937848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:17.938123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:17.938395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:17.970825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d29e09-2321-4b7e-b62d-e0bb68fef725 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.004533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.005167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.005444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.005713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.006420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:18.006521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.006716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.008677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.009132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.017101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 541/1072] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.017887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.041673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.111783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 532/1073] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:18.111988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.112607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.125423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661f6b6-9302-4f93-881a-15aac479959a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.156913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.157573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.157850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.158122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.158907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:18.159214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.160623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.178531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 542/1074] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.179344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.216178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfd191-1079-4a53-977e-200f9288c6c2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.244526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.250333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.250967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.253540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.253540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] PATH_INFO: `/v3/users/cdd18741d6e74152803a4dd85a3bda17` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.253540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd18741d6e74152803a4dd85a3bda17)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.254086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.255679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 533/1075] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/users/cdd18741d6e74152803a4dd85a3bda17 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.256433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.280185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d76df-d3e4-408e-b38e-5189df96f97e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.314299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.314941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.315219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.315494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.325804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.327242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:18.342164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:18.343123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 543/1076] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:18.344030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.350558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.357732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625807-992e-4326-a68d-36ac6aad57d0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.391703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.392281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.392558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.392832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.403279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.404739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:18.419204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:18.420137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 534/1077] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:18.421413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.450655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b7af6-fe20-40e0-ab81-3c99a4cf0c3a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.469755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.484714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.485423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.485711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.485994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.487156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.488640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.490361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 544/1078] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.491120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.520500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca774-cc5d-46c4-abab-7e333399abdf None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.553943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.555078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.555446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.555727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.556479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:18.556785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.558209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.562616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.570511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 535/1079] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.573883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.599185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74699-26c8-4f11-a3e3-b6fa36eead5d None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.642555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.643433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.643875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.644321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.645270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:18.645576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.647033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.661695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 545/1080] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:18.669564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.707420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d22d28-b0d3-4b02-aa5d-cd5fee9b739e None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.720676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.747114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.747789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.748072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.748699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2/users/cdd18741d6e74152803a4dd85a3bda17/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.750352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=96c045ea68c643bc95334d2466df4ba2, user_id=cdd18741d6e74152803a4dd85a3bda17, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.752045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.784817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993294f4-6996-49ed-970b-c63c6084d80f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:18.815922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:18.820742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 536/1081] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:18:18 2022] PUT /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2/users/cdd18741d6e74152803a4dd85a3bda17/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:18:18.823762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:18.824418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:18.824695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:18.824966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:18.832678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:18.834258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:18:18.847488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:18.848927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:18.849859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:18.853174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 546/1082] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:18.855494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:18.967217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013d1a-f711-4615-b1a7-450cfcec79a4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:19.002882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:19.003503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:19.003779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:19.004048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:19.004790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:19.005129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:19.006546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f46cb-f86f-4df1-a11e-945a2480b7de None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:19.018019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 537/1083] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:19.024138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:19.124174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968b10-baf2-41cd-b542-5e16c7ef98e3 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:19.166436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:19.167117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:19.167394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:19.167666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/a8e3f51168f449f2a4fbf199e334ae56` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:19.169836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=e4a3cbcc38d741178be408abd4156959, role_id=a8e3f51168f449f2a4fbf199e334ae56)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:19.171265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95af3ae-255d-4308-95bb-ee8c576bc881 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:19.245342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 547/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:18:19 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/e4a3cbcc38d741178be408abd4156959/roles/a8e3f51168f449f2a4fbf199e334ae56 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:18:20.832002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164b619-4269-4051-bc40-c5e4826e70df None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:20.832537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164b619-4269-4051-bc40-c5e4826e70df None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:20.832537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164b619-4269-4051-bc40-c5e4826e70df None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:20.833925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 538/1085] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:20.839776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:20.840169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:20.840329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:20.858773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:20.859966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:20.906184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:20.927137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 548/1086] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:20.950586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164b619-4269-4051-bc40-c5e4826e70df None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:20.981843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.053256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164b619-4269-4051-bc40-c5e4826e70df None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.088244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.088802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.089111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.089385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.099750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-602d579c-d874-4ec4-9bb4-37aeaa1e5841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:21.105371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:21.106358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:21.107203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 539/1087] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:21.112725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.213840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850926b-865d-47eb-a703-fe7a3f7402df None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.215194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8fec7-2987-4211-8ab5-bf5e053e0d7b None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.215480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8fec7-2987-4211-8ab5-bf5e053e0d7b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.215757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8fec7-2987-4211-8ab5-bf5e053e0d7b None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.217337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 540/1088] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:21.222217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.222494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.222766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.241868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:21.243189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:21.251164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.251784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.252066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.252342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.253098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:21.253405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.254775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.263316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 549/1089] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:21.268885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.312029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.312419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 541/1090] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:18:21.336489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.357177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.369768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ba360-4ee1-4769-b861-9ee99b50276f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.405481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.406114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.406389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.406659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] PATH_INFO: `/v3/users/cdd18741d6e74152803a4dd85a3bda17` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.407819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] RBAC: Authorizing `identity:get_user(user_id=cdd18741d6e74152803a4dd85a3bda17)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.409276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.410878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 550/1091] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/users/cdd18741d6e74152803a4dd85a3bda17 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:21.416723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.434733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d6651-1d6e-4563-aa9b-19776bd0c9e8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.470376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.471017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.471303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.471577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.481538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:21.486412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:21.487524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:21.488376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 542/1092] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:21.493578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.516368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a6f20-4138-422e-aeef-406105a985cd None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.527192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.550221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.550840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.551120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.551510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.574372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.592946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043de1a-7904-4f0c-8481-bf0d187f69f5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.620637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.622067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.627022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.627627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.627898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.628167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.628870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:21.629198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.630642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.639197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 543/1093] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:21.644743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.721918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 551/1094] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 14:18:21.728444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.739553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.743227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f5afc-5ff3-42d9-ba86-88d19e525174 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.777286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.778027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.778311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.778583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.789712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.791151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.792848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:21.805741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:21.806719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:21.807661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:21.807661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:21.807661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:21.807661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:21.807661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 544/1095] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:21.813149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.827579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc388f-d4d9-411a-8da0-784ba9df89d6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.864938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.865632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.865911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.866193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.867396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.868828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.870536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 552/1096] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:21.876672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.916196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8575c02-32aa-46c7-b488-e21802d4d1f9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:21.950864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:21.951484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:21.951819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:21.952091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:21.952783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:21.953123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:21.954488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:21.954794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:21.966965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 545/1097] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:21.972514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:21.976094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af5bc-f530-4250-b0bd-31e97e1f7e8f None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.011401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.012018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.012304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.012579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.013395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=96c045ea68c643bc95334d2466df4ba2, user.id=cdd18741d6e74152803a4dd85a3bda17) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:22.013703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.015129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.021235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 553/1098] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/role_assignments?user.id=cdd18741d6e74152803a4dd85a3bda17&scope.project.id=96c045ea68c643bc95334d2466df4ba2&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:22.027307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:22.075907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e1e83-6b9a-46f9-9b12-f6b69e4b5742 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.112378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.113328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.114173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.114446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.122384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.123842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:18:22.137159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:22.138182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:22.139145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 546/1099] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Jul 29 14:18:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:22.144168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:22.250549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b084f-c293-4b75-a0d7-f8eafe7bc7af None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.285749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.286372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.286653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.286931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.287727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:22.288038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.289580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.303553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:22.308205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 554/1100] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Jul 29 14:18:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:22.320487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:22.413316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b60c-b6fb-4f4d-ab09-5f345e5e4416 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.426675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:22.447051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.447667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.447948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.448225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.515314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.516764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.615914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 547/1101] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:22.622421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:22.720575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d56699-b671-48a2-abbb-7dd7364a73f4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.755381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.756020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.756303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.756575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.766543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.767986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:22.782318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:22.783409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:22.789653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:18:22.789653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:22.789653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:18:22.789653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:22.789653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 555/1102] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:22.789942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:22.804938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:22.886077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec9f3b-c252-43e3-a79c-cad50d129978 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:22.919849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:22.920463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:22.920745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:22.921052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:22.921745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:22.922052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:22.923442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:22.935375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 548/1103] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:22.940925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:22.994658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:23.041347 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2809688-4bb5-4cc9-8baf-65e4883a53e7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:23.074772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:23.075409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:23.075693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:23.075969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:23.082508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:23.083947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:18:23.095818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:23.096698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:23.098112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 556/1104] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:18:23.103539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:23.186456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:23.209910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aee85-322c-436f-ae9c-3ff3026074fa None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:23.245328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:23.245880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:23.246158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:23.246430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:23.247178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:23.247482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:23.248904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3d772-2311-4cc1-92a2-3c9026b02b31 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:23.260773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 549/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:23.266749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:23.366293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b1950-ceb8-4519-92d0-dc40cd6b19b4 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:23.378724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:23.400255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:23.400953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:23.401269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:23.401548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:23.402327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] RBAC: Adding query filter params (role.id=a8e3f51168f449f2a4fbf199e334ae56, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=e4a3cbcc38d741178be408abd4156959) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:23.402640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:23.404059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e0bec-deb0-4b44-ada0-9e9fcb9d2ff7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:23.410197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 557/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:23 2022] GET /identity/v3/role_assignments?user.id=e4a3cbcc38d741178be408abd4156959&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=a8e3f51168f449f2a4fbf199e334ae56 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:24.082543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055050f-5e04-44d3-a96b-4f14efeffa5d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:24.083062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055050f-5e04-44d3-a96b-4f14efeffa5d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:24.083177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055050f-5e04-44d3-a96b-4f14efeffa5d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:24.084483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 550/1107] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:24.090524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:24.090957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:24.091063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:24.110362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:24.111574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:24.181600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 558/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:24.206201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055050f-5e04-44d3-a96b-4f14efeffa5d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:24.308798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055050f-5e04-44d3-a96b-4f14efeffa5d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:24.342608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:24.343174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:24.343453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:24.343724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:24.344869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:24.346325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388688a-e530-4b93-8ad9-a4c781eb97e8 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:24.347901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 551/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:24.353668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:24.451472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e974-4697-4804-9dbb-a9e359e73e98 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:24.458548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:24.485016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:24.485629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:24.485909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:24.486184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:24.486868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:24.488300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890dd2e-310a-45e3-8a45-378c737aba92 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:24.564176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 559/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:18:24 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:26.594291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48635d-84f2-4a1c-abd3-757a7493b814 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:26.594841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48635d-84f2-4a1c-abd3-757a7493b814 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:26.594841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48635d-84f2-4a1c-abd3-757a7493b814 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:26.596175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 552/1111] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:26.601408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:26.602057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:26.602057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:26.619879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:26.621089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:26.687737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 560/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:26.712405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48635d-84f2-4a1c-abd3-757a7493b814 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:26.715288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:26.812508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48635d-84f2-4a1c-abd3-757a7493b814 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:26.845784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:26.846341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:26.846616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:26.846887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] PATH_INFO: `/v3/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:26.848290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] RBAC: Authorizing `identity:get_role(role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:26.849661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:26.850971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 553/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:26 2022] GET /identity/v3/roles/2d3d839c7e5143618bb567643903898d => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:26.856914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:26.946836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:26.955274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c45c-0801-4320-a126-76cbd3cdc94f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:26.988926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:26.989614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:26.989894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:26.990167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] PATH_INFO: `/v3/users/ca95919e085f4b7b8dea350f7f49cbf9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:27.004267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] RBAC: Authorizing `identity:get_user(user_id=ca95919e085f4b7b8dea350f7f49cbf9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:27.005775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:27.007387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 561/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:26 2022] GET /identity/v3/users/ca95919e085f4b7b8dea350f7f49cbf9 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:27.013708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:27.082902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:27.111769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a066d-a7b3-4f6f-8329-470d5e93138b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:27.146184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:27.146789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:27.147069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:27.147346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:27.222895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:27.224342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:27.295448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:27.326602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 554/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:27.332491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:27.429050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12b874-5c0f-478c-8af6-ba09d40e2e82 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:27.462135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:27.462757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:27.463038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:27.463313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:27.464453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:27.465911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e580a5-49da-4dfd-8ccb-c01199710abf None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:27.467545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 562/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:27 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:27.473659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:27.571152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead9dfc-e630-4a80-bb28-a07e9d8cd573 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:27.605501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:27.606109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:27.606368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:27.606671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:27.607449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=96c045ea68c643bc95334d2466df4ba2, user.id=ca95919e085f4b7b8dea350f7f49cbf9) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:27.607756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:27.609191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9a3ba-f522-48a8-855c-8269e291ed9e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:27.614670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 555/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:27 2022] GET /identity/v3/role_assignments?user.id=ca95919e085f4b7b8dea350f7f49cbf9&scope.project.id=96c045ea68c643bc95334d2466df4ba2&role.id=2d3d839c7e5143618bb567643903898d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:29.579238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf53-0399-414a-ab56-26d9bef6c072 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:29.579786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf53-0399-414a-ab56-26d9bef6c072 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:29.579786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3faf53-0399-414a-ab56-26d9bef6c072 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:29.581195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 563/1118] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:29.586536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:29.586947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:29.587082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:29.605112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:29.606275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:29.632121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:29.670620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 556/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:29.692652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3faf53-0399-414a-ab56-26d9bef6c072 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:29.711524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:29.792930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3faf53-0399-414a-ab56-26d9bef6c072 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:29.826577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:29.827206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:29.827489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:29.827767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] PATH_INFO: `/v3/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:29.829229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] RBAC: Authorizing `identity:get_role(role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:29.830641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:29.831918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 564/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:29 2022] GET /identity/v3/roles/2d3d839c7e5143618bb567643903898d => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:29.838476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:29.935932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce319771-be30-4734-8a84-e55cf5bf70c7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:29.969089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:29.969717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:29.969993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:29.970265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] PATH_INFO: `/v3/users/ca95919e085f4b7b8dea350f7f49cbf9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:29.971388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] RBAC: Authorizing `identity:get_user(user_id=ca95919e085f4b7b8dea350f7f49cbf9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:29.972798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:29.974378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 557/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:29 2022] GET /identity/v3/users/ca95919e085f4b7b8dea350f7f49cbf9 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:29.980728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:30.063165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:30.077432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127b3ad-71dc-4bb6-bcb6-3995cadcaa88 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:30.110745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:30.111309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:30.111588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:30.111863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:30.113011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:30.114439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0655-0d4e-419e-87f6-638e074e0e68 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:30.116057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 565/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:29 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:30.122042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:30.152828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:30.220267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59cb29-ddcd-4f65-bb3a-d1a3b72d2522 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:30.252728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:30.253415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:30.253695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:30.253976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2/users/ca95919e085f4b7b8dea350f7f49cbf9/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:30.256110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] RBAC: Authorizing `identity:create_grant(project_id=96c045ea68c643bc95334d2466df4ba2, user_id=ca95919e085f4b7b8dea350f7f49cbf9, role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:30.258195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb3653-20a3-4d93-a4d9-cfa4856295ac None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:30.323966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 558/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:18:30 2022] PUT /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2/users/ca95919e085f4b7b8dea350f7f49cbf9/roles/2d3d839c7e5143618bb567643903898d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:18:32.311354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a71c9b-4584-4122-83b9-42f320d09337 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:32.311933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a71c9b-4584-4122-83b9-42f320d09337 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:32.311933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a71c9b-4584-4122-83b9-42f320d09337 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:32.313274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 566/1124] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:32.318324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:32.318770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:32.318876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:32.337312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:32.338487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:32.402759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 559/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:32.425449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a71c9b-4584-4122-83b9-42f320d09337 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:32.524713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a71c9b-4584-4122-83b9-42f320d09337 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:32.558297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:32.558925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:32.559206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:32.559481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] PATH_INFO: `/v3/roles/2d3d839c7e5143618bb567643903898d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:32.561196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] RBAC: Authorizing `identity:get_role(role_id=2d3d839c7e5143618bb567643903898d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:32.562552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:32.563884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 567/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:32 2022] GET /identity/v3/roles/2d3d839c7e5143618bb567643903898d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:32.569743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:32.592475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:32.664517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78ff09-bc04-49e2-aa80-77ccae947ef4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:32.697781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:32.698349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:32.698626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:32.698901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] PATH_INFO: `/v3/users/ca95919e085f4b7b8dea350f7f49cbf9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:32.700050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] RBAC: Authorizing `identity:get_user(user_id=ca95919e085f4b7b8dea350f7f49cbf9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:32.701508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:32.703075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 560/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:32 2022] GET /identity/v3/users/ca95919e085f4b7b8dea350f7f49cbf9 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:32.708741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:32.732908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:32.807720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ce05b-be1a-47dd-84ed-673fd3dc0209 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:32.840870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:32.841504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:32.841781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:32.842054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:32.906630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:32.908069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:32.930813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:33.004012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 568/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:33.010115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:33.097646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:33.107241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e81a3d-8b0c-471c-bda6-0ff3b682f6cb None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:33.140890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:33.141540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:33.141820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:33.142093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] PATH_INFO: `/v3/projects/96c045ea68c643bc95334d2466df4ba2` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:33.143219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] RBAC: Authorizing `identity:get_project(project_id=96c045ea68c643bc95334d2466df4ba2)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:33.144640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9081637-c5e7-4d0c-ba37-7d328b40ba99 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:33.146298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 561/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:33 2022] GET /identity/v3/projects/96c045ea68c643bc95334d2466df4ba2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:33.153075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:33.251266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83679724-3745-404c-8cc8-95d0e80c5dc7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:33.275353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:33.286435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:33.287045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:33.287326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:33.287599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:33.288376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] RBAC: Adding query filter params (role.id=2d3d839c7e5143618bb567643903898d, scope.project.id=96c045ea68c643bc95334d2466df4ba2, user.id=ca95919e085f4b7b8dea350f7f49cbf9) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:33.288822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:33.290353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186e6d-cefc-4066-be30-407d019d4161 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:33.295992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 569/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:33 2022] GET /identity/v3/role_assignments?user.id=ca95919e085f4b7b8dea350f7f49cbf9&scope.project.id=96c045ea68c643bc95334d2466df4ba2&role.id=2d3d839c7e5143618bb567643903898d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:35.284368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672d16-0ce7-4c04-891f-1c7c5301959c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:35.284923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672d16-0ce7-4c04-891f-1c7c5301959c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:35.284923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf672d16-0ce7-4c04-891f-1c7c5301959c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:35.286455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 562/1131] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:35.292234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:35.292668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:35.292797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:35.311420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:35.312612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:35.377926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 570/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:35.401286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf672d16-0ce7-4c04-891f-1c7c5301959c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:35.501437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf672d16-0ce7-4c04-891f-1c7c5301959c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:35.533712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:35.534332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:35.534613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:35.534891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:35.536031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:35.537492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6c3a-994e-43da-b831-a8a180ff46eb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:35.539610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 563/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:35.545513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:35.642139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30aa65-747e-42ac-abe6-7488b294d2cb None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:35.657996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:35.675730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:35.687530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:35.687530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:35.687530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:35.687530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:35.687530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a099e7-cf06-4f9e-91bb-12db17ba7dd6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:35.780292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 571/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:18:35 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:37.802216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8cbd-f7ee-4802-b560-dcf867a8a3e7 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:37.802755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8cbd-f7ee-4802-b560-dcf867a8a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:37.802755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8cbd-f7ee-4802-b560-dcf867a8a3e7 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:37.804088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 564/1135] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:37.809940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:37.810587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:37.810735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:37.829248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:37.830419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:37.894834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 572/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:37.919287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc8cbd-f7ee-4802-b560-dcf867a8a3e7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:37.947101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:38.019791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc8cbd-f7ee-4802-b560-dcf867a8a3e7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:38.053123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:38.053690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:38.053994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:38.054267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:38.055566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:38.057103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d93e28-7d0f-4cdb-901a-7e0bde5ec977 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:38.063029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 565/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:18:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:38.064629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:38.158207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:38.162039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db3d8a-55fd-455e-911a-473036a2f6ab None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:38.196410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:38.197044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:38.197327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:38.197602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:38.198283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:38.199774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f38969-bde5-4921-9223-7acaf53cf1e8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:38.281796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 573/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:18:38 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:40.306491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfed890-0b00-44d6-bc8c-1e5cbe197445 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:40.307013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfed890-0b00-44d6-bc8c-1e5cbe197445 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:40.307120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfed890-0b00-44d6-bc8c-1e5cbe197445 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:40.308385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 566/1139] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:40.313961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:40.314357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:40.314515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:40.333154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:40.334310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:40.401965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 574/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:40.424959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfed890-0b00-44d6-bc8c-1e5cbe197445 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:40.516051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:40.523640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfed890-0b00-44d6-bc8c-1e5cbe197445 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:40.556591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:40.557251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:40.557821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:40.558097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:40.568084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:40.572953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:40.574149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:40.575103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 567/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:40.580034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:40.661747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:40.677615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad9bd9-fc57-43e9-9978-d117d219b31e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:40.710728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:40.711901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:40.712179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:40.712456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:40.713190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:40.713492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:40.714846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:40.722585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 575/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:40.728613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:40.825843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e53ee-aa1c-4f22-91d6-5416918c76d9 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:40.858422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:40.859033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:40.859306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:40.859575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] PATH_INFO: `/v3/users/2cbe15711bf447d3be76213ae4345872` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:40.872309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] RBAC: Authorizing `identity:get_user(user_id=2cbe15711bf447d3be76213ae4345872)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:40.873756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:40.875325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 568/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:40 2022] GET /identity/v3/users/2cbe15711bf447d3be76213ae4345872 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:40.880891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:41.058010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690d3f-1c3a-4f70-8d16-d52ab77112f6 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:41.106994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:41.107698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:41.108952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:41.109275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:41.133526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:41.193475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:41.196639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:41.310113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 576/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:41.316132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:41.332344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:41.420610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a62a-dd47-4bb3-87e3-da285ef72250 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:41.455779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:41.456417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:41.456699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:41.456977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] PATH_INFO: `/v3/projects/39f584f571184102955a36bdff6bd242` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:41.458164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] RBAC: Authorizing `identity:get_project(project_id=39f584f571184102955a36bdff6bd242)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:41.459605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dcfdb-9e86-4df8-ab7d-b670e8dde386 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:41.462429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 569/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:41 2022] GET /identity/v3/projects/39f584f571184102955a36bdff6bd242 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:41.468494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:41.516192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:41.573151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a9100-bbbe-43e6-8d86-73cd90aac0eb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:41.607454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:41.608071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:41.608397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:41.608674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:41.609489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=39f584f571184102955a36bdff6bd242, user.id=2cbe15711bf447d3be76213ae4345872) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:41.609794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:41.611223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62be6-0e6b-4cbd-9457-10a6776c4806 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:41.616719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 577/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:41 2022] GET /identity/v3/role_assignments?user.id=2cbe15711bf447d3be76213ae4345872&scope.project.id=39f584f571184102955a36bdff6bd242&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:43.608893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206ca6-ecc4-45c7-98af-7bb5d832761d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:43.609483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206ca6-ecc4-45c7-98af-7bb5d832761d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:43.609483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206ca6-ecc4-45c7-98af-7bb5d832761d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:43.610835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 570/1147] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:43.615990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:43.616389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:43.616534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:43.634580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:43.635752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:43.702148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 578/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:43.724444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49206ca6-ecc4-45c7-98af-7bb5d832761d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:43.804895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:43.825058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49206ca6-ecc4-45c7-98af-7bb5d832761d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:43.858478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:43.859157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:43.859385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:43.859656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:43.869552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:43.874387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:43.875564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:43.876614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 571/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:43.881467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:43.922343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:43.979437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88429bab-ec65-4529-aa17-9ef55b1bc721 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:44.012500 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17943f70-659c-4655-8195-278c17356427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:44.013135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943f70-659c-4655-8195-278c17356427 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:44.013418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943f70-659c-4655-8195-278c17356427 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:44.013693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943f70-659c-4655-8195-278c17356427 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:44.014400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17943f70-659c-4655-8195-278c17356427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:44.014701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17943f70-659c-4655-8195-278c17356427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:44.016067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17943f70-659c-4655-8195-278c17356427 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:44.024637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 579/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:44.030316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:44.127957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342dab2-f349-4b49-9d3e-7c8dd189e4a2 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:44.160548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:44.161212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:44.161490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:44.161762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] PATH_INFO: `/v3/users/2cbe15711bf447d3be76213ae4345872` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:44.162905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] RBAC: Authorizing `identity:get_user(user_id=2cbe15711bf447d3be76213ae4345872)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:44.164353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:44.166048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 572/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:44 2022] GET /identity/v3/users/2cbe15711bf447d3be76213ae4345872 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:44.171647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17943f70-659c-4655-8195-278c17356427 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:44.270006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17943f70-659c-4655-8195-278c17356427 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:44.271507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:44.302438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:44.303061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:44.303340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:44.303613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] PATH_INFO: `/v3/projects/39f584f571184102955a36bdff6bd242` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:44.305302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] RBAC: Authorizing `identity:get_project(project_id=39f584f571184102955a36bdff6bd242)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:44.306723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b57e95-ca9b-450c-bf1b-d68de163f524 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:44.309110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 580/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:44 2022] GET /identity/v3/projects/39f584f571184102955a36bdff6bd242 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:44.314813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:44.326591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:44.411749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3657ae1-c982-4bc2-a0b3-74e2188e96d4 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:44.443892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:44.444532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:44.444807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:44.445427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] PATH_INFO: `/v3/projects/39f584f571184102955a36bdff6bd242/users/2cbe15711bf447d3be76213ae4345872/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:44.447320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=39f584f571184102955a36bdff6bd242, user_id=2cbe15711bf447d3be76213ae4345872, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:44.448725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c7be-d674-4bce-8f0c-33b2feee2bd2 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:44.541869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 573/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:18:44 2022] PUT /identity/v3/projects/39f584f571184102955a36bdff6bd242/users/2cbe15711bf447d3be76213ae4345872/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:18:46.502747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157c82a-2330-4c57-996d-23075cd6dd36 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:46.503249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157c82a-2330-4c57-996d-23075cd6dd36 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:46.503249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157c82a-2330-4c57-996d-23075cd6dd36 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:46.504559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 581/1154] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:18:46.510348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1160e-faba-46da-9506-83e264271273 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:46.510748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1160e-faba-46da-9506-83e264271273 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:46.510905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1160e-faba-46da-9506-83e264271273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:46.529295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-68e1160e-faba-46da-9506-83e264271273 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:46.530480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-68e1160e-faba-46da-9506-83e264271273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:46.591222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:46.596233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 574/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:46.619555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157c82a-2330-4c57-996d-23075cd6dd36 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:46.720524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157c82a-2330-4c57-996d-23075cd6dd36 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:46.753132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:46.753694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:46.753971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:46.754246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:46.765144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:46.769950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:46.771022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:46.772047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 582/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:46.777768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1160e-faba-46da-9506-83e264271273 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:46.873598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1160e-faba-46da-9506-83e264271273 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:46.907500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:46.908109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:46.908387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:46.908660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:46.909516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:46.909818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:46.911186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:46.919032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 575/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:46.924635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:46.956246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:47.025522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca73c-d536-40ee-b677-4287ebacddef None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:47.058153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:47.058774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:47.059051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:47.059325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] PATH_INFO: `/v3/users/2cbe15711bf447d3be76213ae4345872` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:47.060723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] RBAC: Authorizing `identity:get_user(user_id=2cbe15711bf447d3be76213ae4345872)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:47.062184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:47.063779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 583/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:18:46 2022] GET /identity/v3/users/2cbe15711bf447d3be76213ae4345872 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:47.069445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:47.126994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:47.167271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f48ec-2691-4b68-93ae-a333b32d47dc None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:47.199667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:47.200277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:47.200555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:47.200829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:47.266760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:47.268200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:47.324196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:47.364868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 576/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:47.370538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:47.466298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:47.469980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c51d2a-a5e8-43f3-ba0d-93cccd05e491 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:47.502452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:47.503076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:47.503627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:47.503627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] PATH_INFO: `/v3/projects/39f584f571184102955a36bdff6bd242` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:47.504731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] RBAC: Authorizing `identity:get_project(project_id=39f584f571184102955a36bdff6bd242)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:47.506188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea486f8-05c0-42c7-ace7-b628a6e27194 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:47.508498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 584/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:18:47 2022] GET /identity/v3/projects/39f584f571184102955a36bdff6bd242 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:47.514431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:47.610306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e3706-583f-403d-8667-bb6a8d0a683f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:47.643255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:47.643859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:47.644135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:47.644406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:47.645209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=39f584f571184102955a36bdff6bd242, user.id=2cbe15711bf447d3be76213ae4345872) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:47.645517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:47.646922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893be902-6316-447a-9858-0b08d4e91a24 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:47.653118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 577/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:18:47 2022] GET /identity/v3/role_assignments?user.id=2cbe15711bf447d3be76213ae4345872&scope.project.id=39f584f571184102955a36bdff6bd242&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:49.645979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7bba-d152-4e2d-b4f0-45aade47976e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:49.646541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7bba-d152-4e2d-b4f0-45aade47976e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:49.646541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7bba-d152-4e2d-b4f0-45aade47976e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:49.647882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 585/1162] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:49.653085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:49.653473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:49.653672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:49.671214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:49.672386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:49.690098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:49.738225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 578/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:49.760564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae7bba-d152-4e2d-b4f0-45aade47976e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:49.861125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae7bba-d152-4e2d-b4f0-45aade47976e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:49.894153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:49.894761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:49.895037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:49.895309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:49.927948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:49.961877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:49.963313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:50.058142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 586/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:18:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:18:50.064308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:50.167252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce978d2c-93d1-4b4b-950f-12f28e36c4e1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:50.201525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:50.202094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:50.202372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:50.202646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:50.209088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:50.210519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c23d94a4-0834-4b9f-a152-1506e5ec2cbe None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:50.222614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:50.223800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:18:50.224669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 579/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:18:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:50.230869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:50.295272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:50.328571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3ed9-8013-4354-830a-3b88af983b0c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:50.363755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:50.364366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:50.364649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:50.364926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:50.365650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:18:50.365956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:50.367478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac89b7e-3727-4985-8525-2c5c0822c7ff None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:50.374623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 587/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Jul 29 14:18:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:52.324781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6c20e-2d26-49f3-9f42-6bf99d68af9b None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:52.325327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6c20e-2d26-49f3-9f42-6bf99d68af9b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:52.325435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6c20e-2d26-49f3-9f42-6bf99d68af9b None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:52.326734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 580/1167] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:52.331727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8357-27ab-4f15-8dec-4c86ffdc4e4a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:52.332175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8357-27ab-4f15-8dec-4c86ffdc4e4a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:52.332270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8357-27ab-4f15-8dec-4c86ffdc4e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:52.350826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ee8b8357-27ab-4f15-8dec-4c86ffdc4e4a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:52.352748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ee8b8357-27ab-4f15-8dec-4c86ffdc4e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:52.418094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 588/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:52.439969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6c20e-2d26-49f3-9f42-6bf99d68af9b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:52.455409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:52.539733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6c20e-2d26-49f3-9f42-6bf99d68af9b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:52.572627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:52.573264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:52.573544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:52.573817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:52.574524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:52.575878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424a08c-1692-4e84-a3b6-b0a5b05df149 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:52.645246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 581/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Fri Jul 29 14:18:52 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:54.639584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6b83f-f168-4baa-b96b-044c143c5d42 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:54.640112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6b83f-f168-4baa-b96b-044c143c5d42 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:54.640112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6b83f-f168-4baa-b96b-044c143c5d42 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:54.641520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 589/1170] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:54.646705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:54.647083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:54.647269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:54.666894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:54.668076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:54.732896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 582/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:54.755691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6b83f-f168-4baa-b96b-044c143c5d42 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:54.815992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:54.856398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6b83f-f168-4baa-b96b-044c143c5d42 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:54.889796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:54.890365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:54.890641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:54.890911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] PATH_INFO: `/v3/domains/fed55e6979174a2bbb7037f3161b2590` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:54.901225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=fed55e6979174a2bbb7037f3161b2590)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:54.902667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6533c-5946-4c2f-afb2-e1f5a22c33ca None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:54.904252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 590/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Jul 29 14:18:54 2022] GET /identity/v3/domains/fed55e6979174a2bbb7037f3161b2590 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:18:54.910055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:54.913081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:55.006180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d0bc6-9257-4323-a646-913a42f7c9ae None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:55.038627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:55.039782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:55.040026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:55.040295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:55.040959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:55.042408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d022b1e-b0de-42fa-85f6-47c835437174 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:55.130693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 583/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:18:54 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:18:57.137270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90172e-621a-49a2-9237-34bd4ffd5ac1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:57.137825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90172e-621a-49a2-9237-34bd4ffd5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:57.137825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90172e-621a-49a2-9237-34bd4ffd5ac1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:57.139161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 591/1174] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:57.144257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:57.144638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:57.144812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:57.163220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:57.165226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:57.214340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:57.230072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 584/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:57.254053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90172e-621a-49a2-9237-34bd4ffd5ac1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:57.355594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90172e-621a-49a2-9237-34bd4ffd5ac1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:57.375143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:57.389354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:57.390001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:57.390284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:57.390561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] PATH_INFO: `/v3/domains/fed55e6979174a2bbb7037f3161b2590` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:57.391717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fed55e6979174a2bbb7037f3161b2590)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:57.393178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc94ff4-7978-4ea8-a357-5016969c8208 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:57.394778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 592/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Jul 29 14:18:57 2022] GET /identity/v3/domains/fed55e6979174a2bbb7037f3161b2590 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:18:57.401649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:57.497977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb876-b0e4-4c16-ba35-33c4cbd31d5d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:57.531705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:57.532300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:57.532577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:57.532846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:57.533547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:18:57.534957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8973d8-c44c-451d-b52a-b70aa17f5bce None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:18:57.630817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 585/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:18:57 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:18:59.644923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5370-3203-4a32-8ead-67b1d2c52892 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:59.645485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5370-3203-4a32-8ead-67b1d2c52892 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:59.645485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5370-3203-4a32-8ead-67b1d2c52892 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:59.646878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 593/1178] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:18:59.652074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:59.652494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:59.652629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:59.670746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:18:59.671921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:18:59.706653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:18:59.736640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 586/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:18:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:18:59.760249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5370-3203-4a32-8ead-67b1d2c52892 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:18:59.861067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5370-3203-4a32-8ead-67b1d2c52892 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:18:59.893238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:18:59.893865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:18:59.894145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:18:59.894420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:18:59.905660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:18:59.910385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:18:59.911584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:18:59.912641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 594/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:18:59.918295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:00.013816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541cd98-be9a-4d53-94dd-eac69041daeb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:00.047418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:00.048025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:00.048303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:00.048578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:00.049320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:19:00.049625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:00.050985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:00.058871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 587/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:00.064723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:00.113796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:00.164925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839120-31a2-4ea1-9f01-edb8251bf6ef None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:00.198212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:00.198840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:00.199119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:00.199396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] PATH_INFO: `/v3/users/3419c21d70964e2fba0f141fa4bff89e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:00.213076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] RBAC: Authorizing `identity:get_user(user_id=3419c21d70964e2fba0f141fa4bff89e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:00.214522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:00.216124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 595/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:19:00 2022] GET /identity/v3/users/3419c21d70964e2fba0f141fa4bff89e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:00.221827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:00.256165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:00.321534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b7da5-f50e-4859-abeb-d2d9b0145a80 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:00.354379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:00.354984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:00.355267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:00.355545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:00.420040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:00.421449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:00.453145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:00.517536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 588/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:19:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:19:00.523345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:00.621217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29470730-c2b7-4ae1-a8f9-e7e1f0043a81 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:00.629070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:00.655741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:00.656476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:00.657074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:00.657350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] PATH_INFO: `/v3/projects/51c577d781d64ab7850fbdbd3697bf8a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:00.658490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] RBAC: Authorizing `identity:get_project(project_id=51c577d781d64ab7850fbdbd3697bf8a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:00.659906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169ae2d-e1f3-4831-ac57-4bddb8e9140c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:00.661562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 596/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:19:00 2022] GET /identity/v3/projects/51c577d781d64ab7850fbdbd3697bf8a => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:00.667637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:00.763921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de3bbd-95de-4278-816e-f2fb51689432 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:00.796341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:00.796941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:00.797247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:00.797520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:00.798296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=51c577d781d64ab7850fbdbd3697bf8a, user.id=3419c21d70964e2fba0f141fa4bff89e) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:19:00.798602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:00.800016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246e395-9d9b-4eef-b7f6-e161e05df71e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:00.801778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:00.806154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 589/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:19:00 2022] GET /identity/v3/role_assignments?user.id=3419c21d70964e2fba0f141fa4bff89e&scope.project.id=51c577d781d64ab7850fbdbd3697bf8a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:02.745565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd49b7-088a-41a4-9e68-c59d0bb83c2e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:02.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd49b7-088a-41a4-9e68-c59d0bb83c2e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:02.746144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd49b7-088a-41a4-9e68-c59d0bb83c2e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:02.747448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 597/1186] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:19:02.752522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:02.752961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:02.753093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:02.771247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:19:02.772408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:19:02.837873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 590/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:19:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 29 14:19:02.860389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd49b7-088a-41a4-9e68-c59d0bb83c2e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:02.959237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd49b7-088a-41a4-9e68-c59d0bb83c2e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:02.993627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:02.994255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:02.994533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:02.994806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:03.004637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:19:03.009588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:19:03.010791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:19:03.011772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 598/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:19:03.016542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:03.114055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516b0f8-aa17-40a0-a947-1991d52235a6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:03.139151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:03.146569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:03.147182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:03.147458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:03.147731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:03.148439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:19:03.148744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:03.151000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:03.163731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 591/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:03.165596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:03.200272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:03.264011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad138-72b7-41f6-9fbd-626d35270124 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:03.297154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:03.297779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:03.298058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:03.298329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] PATH_INFO: `/v3/users/3419c21d70964e2fba0f141fa4bff89e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:03.299476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] RBAC: Authorizing `identity:get_user(user_id=3419c21d70964e2fba0f141fa4bff89e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:03.300912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:03.302514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 599/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:19:03 2022] GET /identity/v3/users/3419c21d70964e2fba0f141fa4bff89e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:03.308285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:03.413823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393def5f-8866-4465-a7e0-422938ba28a5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:03.446369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:03.447077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:03.447355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:03.447626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] PATH_INFO: `/v3/projects/51c577d781d64ab7850fbdbd3697bf8a` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:03.448763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] RBAC: Authorizing `identity:get_project(project_id=51c577d781d64ab7850fbdbd3697bf8a)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:03.450223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53cb0-2cb6-466d-9e88-a7c658030de9 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:03.451868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 592/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:19:03 2022] GET /identity/v3/projects/51c577d781d64ab7850fbdbd3697bf8a => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:03.457825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:03.554240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72b3c9-1ac4-4733-90b4-72c8aceda43b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:03.556269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:03.589774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:03.590427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:03.590708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:03.590982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] PATH_INFO: `/v3/projects/51c577d781d64ab7850fbdbd3697bf8a/users/3419c21d70964e2fba0f141fa4bff89e/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:03.592900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] RBAC: Authorizing `identity:create_grant(project_id=51c577d781d64ab7850fbdbd3697bf8a, user_id=3419c21d70964e2fba0f141fa4bff89e, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:03.594355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a251672-5e61-4f6f-9a09-180a10a6ea53 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:03.678417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 600/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:19:03 2022] PUT /identity/v3/projects/51c577d781d64ab7850fbdbd3697bf8a/users/3419c21d70964e2fba0f141fa4bff89e/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:19:05.640156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55009-a80c-4ae4-b6ab-b6d1d22ec729 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:05.640682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55009-a80c-4ae4-b6ab-b6d1d22ec729 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:05.640682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b55009-a80c-4ae4-b6ab-b6d1d22ec729 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:05.642074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 593/1193] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:19:05.647091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:05.647511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:05.647638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:05.666260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:19:05.667419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:19:05.733568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 601/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:19:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:19:05.756156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b55009-a80c-4ae4-b6ab-b6d1d22ec729 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:05.815234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:05.856208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b55009-a80c-4ae4-b6ab-b6d1d22ec729 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:05.892745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:05.893875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:05.894479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:05.894854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:05.905636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:19:05.910467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:19:05.911561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:19:05.912703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 594/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:19:05.917928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:06.014465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:06.022896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447951-4a8a-4b9c-b7e3-c7d88f303d49 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:06.057143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:06.059669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:06.059669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:06.059669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:06.059669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:19:06.059669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:06.060647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:06.068683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 602/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:06.075136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:06.174359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c88d9c-2e42-415b-b346-46c5f9af70b6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:06.191564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:06.207212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:06.207841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:06.208115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:06.208384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] PATH_INFO: `/v3/users/3419c21d70964e2fba0f141fa4bff89e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:06.209551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] RBAC: Authorizing `identity:get_user(user_id=3419c21d70964e2fba0f141fa4bff89e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:06.210971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:06.212650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 595/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Jul 29 14:19:06 2022] GET /identity/v3/users/3419c21d70964e2fba0f141fa4bff89e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:06.218388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:06.318303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78994-bb62-4250-af36-6e467b647faf None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:06.350348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:06.350952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:06.351232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:06.351508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:06.375177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:06.417712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:06.419080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:06.513358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 603/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:19:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:19:06.519174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:06.615759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72879a83-af39-495d-8b42-eba0aa9fad6f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:06.647995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:06.648617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:06.648896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:06.649783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] PATH_INFO: `/v3/projects/51c577d781d64ab7850fbdbd3697bf8a` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:06.650931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] RBAC: Authorizing `identity:get_project(project_id=51c577d781d64ab7850fbdbd3697bf8a)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:06.652343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03732eb-a7e3-46bf-8f28-03a2095867ca None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:06.654006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 596/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Jul 29 14:19:06 2022] GET /identity/v3/projects/51c577d781d64ab7850fbdbd3697bf8a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:06.659959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:19:06.714463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:06.758481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb1a0-249e-46db-989d-90e280d94a35 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:19:06.792357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:19:06.792961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:19:06.793270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:19:06.793548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:19:06.794324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=51c577d781d64ab7850fbdbd3697bf8a, user.id=3419c21d70964e2fba0f141fa4bff89e) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:19:06.794638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:19:06.796053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db973612-b4c4-4634-8bc3-34002d1d47eb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:19:06.801731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 604/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:19:06 2022] GET /identity/v3/role_assignments?user.id=3419c21d70964e2fba0f141fa4bff89e&scope.project.id=51c577d781d64ab7850fbdbd3697bf8a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:19:16.643960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=73 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:19:16.799134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=41 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:18.209312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f904aa-16b4-4d0a-878a-0b4903f117de None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:18.209891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f904aa-16b4-4d0a-878a-0b4903f117de None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:18.209891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f904aa-16b4-4d0a-878a-0b4903f117de None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:18.211294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 597/1201] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:18.216280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:18.216748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:18.216851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:18.234882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:18.236065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:18.303790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 605/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:18.328604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f904aa-16b4-4d0a-878a-0b4903f117de None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:18.430174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f904aa-16b4-4d0a-878a-0b4903f117de None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:18.462543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:18.463167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:18.463441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:18.463712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:18.474455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:18.475907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:18.490134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:18.493596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:18.494856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:18.494856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:18.494856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:18.494856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:18.494856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 598/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:18.497265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:18.597217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbc448-f5bc-4a1b-801f-0737298a8e60 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:18.625438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:18.631734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:18.632360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:18.632639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:18.632913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:18.633635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:18.633941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:18.635321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1f082-a796-4a2d-8f61-c7a9a56e0939 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:18.647558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 606/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:18.652888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:18.707454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:18.755260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90dfa8-3dac-4a3b-9317-086082cf25e1 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:18.787933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:18.788528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:18.788803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:18.789098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:18.789766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:18.791188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8a759-c431-48b9-a306-f74c61027be0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:18.873864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 599/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Jul 29 14:20:18 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:20:20.889768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbdd18-5057-481f-b3ac-6ace9da30fa7 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:20.890318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbdd18-5057-481f-b3ac-6ace9da30fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:20.890318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbdd18-5057-481f-b3ac-6ace9da30fa7 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:20.891674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 607/1206] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:20.897686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:20.898114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:20.898246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:20.916246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:20.917472 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:20.984906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 600/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:21.007721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbdd18-5057-481f-b3ac-6ace9da30fa7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.108962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbdd18-5057-481f-b3ac-6ace9da30fa7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.141778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.142368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.142644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.142913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.154179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:21.159000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:21.160019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:21.161572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 608/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:21.166846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.204937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:21.262550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840d174-624f-4ac2-a286-46791f3b2935 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.297785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.298393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.298695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.298964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.299668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:21.299967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:21.301402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:21.308969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 601/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:21.315321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.366477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:21.416136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2f7af-2f66-4db5-9915-cb89bd3187db None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.449752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.450419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.450968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.451459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.463087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:21.464525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:21.478744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:21.479748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:21.481030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:21.481030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:21.481030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:21.481030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:21.481030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 609/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:21.485815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.581189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:21.584572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8b31-487c-4f49-8d54-0a2ec95a4f9d None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.618550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474109f-726b-4724-955c-1d892588e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.619168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474109f-726b-4724-955c-1d892588e922 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.619450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474109f-726b-4724-955c-1d892588e922 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.619723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474109f-726b-4724-955c-1d892588e922 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.620417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474109f-726b-4724-955c-1d892588e922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:21.620722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474109f-726b-4724-955c-1d892588e922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:21.622125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474109f-726b-4724-955c-1d892588e922 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:21.633611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 602/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:21.639286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.748614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb18ff-78ab-47f4-a36c-017b3a95d9e9 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.750081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:21.782999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.783629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.783906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.784177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.792258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:21.793725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:21.805370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:21.807226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:21.808211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 610/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:21.812711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474109f-726b-4724-955c-1d892588e922 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:21.912504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474109f-726b-4724-955c-1d892588e922 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:21.947176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:21.947779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:21.948054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:21.948322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:21.949125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:21.949430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:21.950883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:21.968173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 603/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:21.974076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:22.073956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6c4e-747a-4457-b46c-437d9779b4f8 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:22.106818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:22.107445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:22.107716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:22.107992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:22.138164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:22.175222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:22.177392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:22.275087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 611/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:20:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:20:22.281170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:22.300846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:22.381841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aefe6-e033-4528-a244-d4e78faa3dad None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:22.416278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:22.416901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:22.417204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:22.417474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:22.427955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:22.429441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:22.444498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:22.445584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:22.446729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:22.446729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:22.446729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:22.446729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:22.446729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 604/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:22.451597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:22.511901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:22.549309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa9ad4-36b8-4ac0-b324-09b2197e5e9a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:22.584114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:22.584722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:22.585014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:22.585397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:22.586073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:22.586372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:22.587735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:22.599486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 612/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:22.605518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:22.673728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:22.703647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb722ae-826f-447e-a77d-55af646a6a0f None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:22.737523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:22.738156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:22.738436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:22.738768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:22.745620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:22.747053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:22.758224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:22.759246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:22.760625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 605/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:22.765456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:22.866069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ee427-6904-4905-aab7-560a709bb190 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:22.888638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:22.900534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:22.901198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:22.901483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:22.901759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:22.902507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:22.902815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:22.904243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066236a-db6f-4c92-8b15-d0f837946df3 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:22.916064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 613/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:22.922213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:23.022346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0ffc8-59ff-4e1a-9b3e-3df63852a0fa None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:23.049849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:23.055192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:23.055798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:23.056075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:23.056350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:23.057162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=60aee4f7a30845f69db2e71f02fa139b) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:23.060837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:23.060837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43860fd9-f06a-47a2-986c-33113b34616a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:23.064297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 606/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:20:22 2022] GET /identity/v3/role_assignments?user.id=60aee4f7a30845f69db2e71f02fa139b&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:25.057727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3f85-b37d-43f5-b36a-9c006e06c36d None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.058285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3f85-b37d-43f5-b36a-9c006e06c36d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.058285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f3f85-b37d-43f5-b36a-9c006e06c36d None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.059652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 614/1220] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:25.065232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.065646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.065780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.085300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:25.086470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:25.152442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 607/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:25.174610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f3f85-b37d-43f5-b36a-9c006e06c36d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:25.275838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f3f85-b37d-43f5-b36a-9c006e06c36d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:25.308626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:25.309219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.309501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.309776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.320678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:25.326120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:25.327117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:25.327963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 615/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:25.333343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:25.431582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fb52-8138-4afc-99c4-d25b930f15bb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:25.455515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:25.466423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:25.467078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.467397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.467670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.468379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:25.468688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:25.470103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:25.478581 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 608/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:25.484519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:25.528641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:25.583514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52023f2-d5dd-472d-ada0-4fc1a28f74c7 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:25.617886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:25.618519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.618794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.619066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.629131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:25.632080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:25.645990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:25.647005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:25.647909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:25.647909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:25.647909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:25.647909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:25.647909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 616/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 29 14:20:25.653162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:25.752524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9972b-2d69-4c6a-bc7b-2228de480b16 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:25.785453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:25.786767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.787058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.787326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.788009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:25.788310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:25.789684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:25.801447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 609/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:25.807698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:25.904126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:25.910957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4cc1c-95fe-4fd2-b86a-b80818af4d18 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:25.944384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:25.945040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:25.945324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:25.945598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:25.952822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:25.954286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:25.965901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:25.967006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:25.968033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 617/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:25.972840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.019791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:26.070930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e6ab6-727f-4e77-abca-58d336c55c76 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.104636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.105280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.105564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.105842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.106633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:26.106948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.108362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.125086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 610/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:26.131451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.233026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca42b3-4a40-4c05-91f8-aae797d61965 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.266856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.267491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.267770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.268040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.278362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.279794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:26.293744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:26.294815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:26.295758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:26.295758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:26.295758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:26.295758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:26.295758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 618/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:26.300661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.399045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61b17-ee8e-49aa-a83f-2cb9ed12b434 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.401427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:26.432135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.432739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.433085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.433355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.434036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:26.434335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.435679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.447293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 611/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:26.452920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.462831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:26.553548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7b71-9aa5-4226-998b-a3b640209484 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.585559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.586190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.586470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.586742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.593152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.594595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:26.606180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:26.607227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:26.608545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 619/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:26.614548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.711562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d496d7f-b16e-41c3-b6b1-382b7a9a8248 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.745999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34367472-b524-40d7-a995-2143561b79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.746607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34367472-b524-40d7-a995-2143561b79ce None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.746889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34367472-b524-40d7-a995-2143561b79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.747161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34367472-b524-40d7-a995-2143561b79ce None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.747914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34367472-b524-40d7-a995-2143561b79ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:26.748225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34367472-b524-40d7-a995-2143561b79ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.749678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34367472-b524-40d7-a995-2143561b79ce None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.762284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 612/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:26.768270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:26.835229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:26.869645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586802-eea8-4337-ba27-f24ef9001cb2 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:26.902758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:26.903418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:26.907595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:26.907595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/60aee4f7a30845f69db2e71f02fa139b/roles/daaab464c0214e7bbb3528ad80727dad` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:26.918155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=60aee4f7a30845f69db2e71f02fa139b, role_id=daaab464c0214e7bbb3528ad80727dad)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:26.919639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f6b94-98bc-4427-96e6-55d10d5908ba None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:26.994952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 620/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:20:26 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/60aee4f7a30845f69db2e71f02fa139b/roles/daaab464c0214e7bbb3528ad80727dad => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:20:28.972781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a776-ca71-4597-b3a8-e2855cfed402 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:28.973338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a776-ca71-4597-b3a8-e2855cfed402 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:28.973570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855a776-ca71-4597-b3a8-e2855cfed402 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:28.974930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 613/1233] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:28.980088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:28.980507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:28.980641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:28.999051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:29.000216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:29.067451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 621/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:29.091349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855a776-ca71-4597-b3a8-e2855cfed402 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.116677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:29.195477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855a776-ca71-4597-b3a8-e2855cfed402 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:29.228412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:29.228972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:29.229279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:29.229548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:29.240150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:29.245029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:29.246124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:29.247114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 614/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:29.252214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.298127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:29.349721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9702ad-5c21-4b14-81eb-406f0e20ffb8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:29.383485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:29.384102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:29.384381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:29.384654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:29.385429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:29.385752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:29.387110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:29.394874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 622/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:29.400946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.485394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:29.501071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f57e05-4913-4f32-b9ba-ba167f54bb7b None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:29.533161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:29.533793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:29.534072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:29.534343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:29.545537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:29.546968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:29.561473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:29.562838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:29.563749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:29.563749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:29.563749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:29.563749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:29.563749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 615/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:29.569107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.670067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633cc9b-4ef8-4b1a-b4a0-e4316a558e71 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:29.679596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:29.705650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:29.706262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:29.706543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:29.706819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:29.707534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:29.707838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:29.709266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:29.720785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 623/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:29.727340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.826034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634085-1367-40f3-8019-c3726f8e8b06 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:29.858835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:29.859461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:29.859741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:29.860065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:29.862695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:29.868715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:29.870945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:29.881382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:29.882504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:29.883656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 616/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:29.888412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:29.991413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29840d8-b851-4447-961e-b0bed14b0291 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.025944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.026561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.026846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.027121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.027907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:30.028217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.029673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.047555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 624/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:30.053452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:30.154162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c43cc-5bf1-45ce-b8f0-8f9f92517cb5 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.188418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.189058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.189338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.191274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.249411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:30.256231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.257733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.355122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 617/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:20:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 29 14:20:30.360925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:30.390198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:30.462031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54d23-3bf0-4c28-8b1d-4110c63f5f49 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.494292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.494919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.495196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.495469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.505871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.507304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:30.524038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:30.525165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:30.526285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:30.526285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:30.526285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:30.526285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:30.526285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 625/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:30.530919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:30.623771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:30.630000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e1d2c-0f8e-4cb2-9aa0-8ef4bac030f3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.664291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.664907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.665217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.665494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.666182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:30.666482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.667861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.679370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 618/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:30.685040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:30.763403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:30.784192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0548a-9dc9-4a9b-9635-54223abe7893 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.818034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.818665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.818942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.819214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.825746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.827214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:30.838546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:30.839628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:30.840593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 626/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:30.845611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:30.946129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4135-a317-408e-8b13-d3e969eb8e85 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:30.978783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:30.979400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:30.979681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:30.979980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:30.980729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:30.981072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:30.982492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb16b5-db57-4e4a-868f-6b7671fecc02 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:30.995201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 619/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:31.001345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:31.102252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f94b3-eec4-4d02-b1c5-45692f3f95bb None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:31.136045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:31.136655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:31.136935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:31.137268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:31.138052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] RBAC: Adding query filter params (role.id=daaab464c0214e7bbb3528ad80727dad, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=60aee4f7a30845f69db2e71f02fa139b) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:31.138385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:31.139869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704ba5f-3cb9-4f7a-bac8-e6dc18be751f None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:31.145885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 627/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:20:31 2022] GET /identity/v3/role_assignments?user.id=60aee4f7a30845f69db2e71f02fa139b&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=daaab464c0214e7bbb3528ad80727dad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:33.211153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420876-87e8-43bc-b02d-389080b42167 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.211672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420876-87e8-43bc-b02d-389080b42167 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.211760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420876-87e8-43bc-b02d-389080b42167 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.213099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 620/1247] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:33.218184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.218583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.218734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.226155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:33.238344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:33.239740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:33.306398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 628/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:33.333842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420876-87e8-43bc-b02d-389080b42167 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:33.353154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:33.435391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420876-87e8-43bc-b02d-389080b42167 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:33.470731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:33.471354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.471630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.471901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.482288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:33.487262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:33.488294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:33.496393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 621/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:33.497311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:33.593075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a65f-bfd0-4c81-a55f-e754f2cbeb0d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:33.624194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:33.629008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:33.630317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.630601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.630868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.631580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:33.631883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:33.633270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:33.650393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 629/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:33.656112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:33.732085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:33.755616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69b7e-e573-4d80-a3db-7fa2f953a042 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:33.788305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:33.788938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.789245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.789516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.799798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:33.801814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:33.816099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:33.817747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:33.818663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:33.818663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:33.818663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:33.818663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:33.818663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 622/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:33.824036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:33.924710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e1bb-4929-44af-b906-62cf8ad69c9c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:33.958745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:33.959354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:33.959640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:33.959919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:33.960608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:33.960919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:33.962321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:33.974505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 630/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:33.980395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:34.081215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cabf7-bff9-4811-9f49-a1c0c8d8f4a7 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:34.107459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:34.115743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:34.116376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:34.116657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:34.116932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:34.124281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:34.125805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:34.136822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:34.137983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:34.139169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 623/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:34.143943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:34.204546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:34.248508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf10b-e62e-415e-ae3e-8256531c05fe None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:34.281885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:34.282500 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:34.282784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:34.283060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:34.283850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:34.284156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:34.285617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:34.304501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 631/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:34.310385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:34.409590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f277b1-8fce-4bea-9f94-c9c562cfe293 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:34.443222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:34.443825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:34.444126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:34.444401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:34.499236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:34.511269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:34.512713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:34.610087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 624/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:20:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:20:34.616578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:34.717310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948945-3afb-4f1a-a102-b0f0618ad425 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:34.730675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:34.750882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:34.751512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:34.751792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:34.752068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:34.761954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:34.763397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:34.778409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:34.779504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:34.780789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:34.780789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:34.780789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:34.780789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:34.780789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 632/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:34.785417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:34.872664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:34.885088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082aede-9639-472b-8067-5285a8209af3 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:34.918639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:34.919255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:34.919536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:34.919811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:34.920501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:34.920806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:34.922199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:34.933069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 625/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:34.938993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:35.040108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ef1b0-c65a-40d9-bf5a-56f1c3082f8e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:35.074476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:35.075111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:35.075391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:35.075664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:35.082182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:35.091946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:35.095354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:35.096366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:35.097591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 633/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:35.102377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:35.203306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297360-088b-46cb-bb84-d4456a3dcac0 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:35.238090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:35.238696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:35.239590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:35.239800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:35.240535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:35.240834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:35.242281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7618bd0-c145-418e-a489-4c7255cbf8eb None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:35.253870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 626/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:35.260624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:35.287003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:35.360319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeea2c8-1def-4df5-a55d-4d2d64b8ace1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:35.395514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:35.396145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:35.396426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:35.396699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:35.397512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=60aee4f7a30845f69db2e71f02fa139b) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:35.397825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:35.399305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4ac50-329e-4028-9c0e-2d4ec4665950 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:35.404881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 634/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:20:35 2022] GET /identity/v3/role_assignments?user.id=60aee4f7a30845f69db2e71f02fa139b&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:20:37.388204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ae9b-a70b-4a76-a8c0-aacb82294980 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:37.388764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ae9b-a70b-4a76-a8c0-aacb82294980 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:37.388764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863ae9b-a70b-4a76-a8c0-aacb82294980 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:37.390117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 627/1261] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:37.395511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:37.395922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:37.396065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:37.415518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:37.416643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:37.484521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 635/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:37.506415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863ae9b-a70b-4a76-a8c0-aacb82294980 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:37.517070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:37.607937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863ae9b-a70b-4a76-a8c0-aacb82294980 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:37.642797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:37.643432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:37.643706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:37.643973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:37.654953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:37.659761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:37.661600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:37.662891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 628/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:37.668599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:37.687131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:37.769482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524c0fb-8d1d-4706-a4e6-3869e843f896 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:37.805352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:37.805964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:37.806239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:37.806507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:37.807214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:37.807512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:37.808863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:37.817692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 636/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:37.823538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:37.890415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:37.924290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b05b91-c8dc-4bcb-a667-f58b42471337 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:37.957160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:37.958441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:37.958725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:37.958998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:37.970191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:37.971648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:37.985730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:37.986774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:37.987743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:37.987743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:37.987743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:37.987743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:37.987743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 629/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:37.992843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.074113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:38.093846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21744b6-3101-4bd9-9988-e2715bf0d521 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.127390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.127996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.128272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.128542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.129252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:38.130468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.131840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.143125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 637/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:38.149647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.249083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09c846-1a0b-47d3-b4db-2eb6189af3fa None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.269435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:38.283360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.283989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.284263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.284532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.292144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.295444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:38.305495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:38.306628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:38.307659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 630/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:38.312760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.411699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee06f7-05fd-4ff2-94da-84d24a5eea6c None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.446152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.446765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.447045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.447317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.448112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:38.448416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.449876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.458570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:38.466703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 638/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:38.472496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.573255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c7c2f-19b6-4364-b7e7-9d09bee1ada6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.607808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.608446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.608723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.609016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.620372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.621844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:38.635697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:38.636846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:38.638170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:38.638170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:38.638170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:38.638170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:38.638170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 631/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:38.642769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.745011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330cc3-3a24-45fe-993d-749989607f58 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.779936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.780541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.780820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.781122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.781810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:38.782112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.783576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.795080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 639/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:38.800826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:38.834755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:38.903553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2b56-ffca-4986-bfc1-3957343f269a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:38.936686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:38.937341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:38.937876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:38.938150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:38.945223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:38.946659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:38.957753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:38.959185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:38.960294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 632/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:38.964917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:39.026037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:39.066671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750c45-6bca-4c67-9cc2-3cb8ce80cbfc None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:39.099345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:39.099893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:39.100174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:39.100447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:39.101215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:39.101523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:39.102993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e37aac-3133-4e5f-8b82-d248e655a610 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:39.115751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 640/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:39.121844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:39.217585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:39.225971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2663-8250-455d-80a3-4e1104231ced None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:39.259138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:39.259806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:39.260086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:39.260362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] PATH_INFO: `/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/60aee4f7a30845f69db2e71f02fa139b/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:39.262310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] RBAC: Authorizing `identity:create_grant(project_id=77d5d61ad1ff4826b55380d74a48285a, user_id=60aee4f7a30845f69db2e71f02fa139b, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:39.263754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c58fe6-1f77-4d2a-ab01-8333cee5d035 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:39.342889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 633/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Jul 29 14:20:39 2022] PUT /identity/v3/projects/77d5d61ad1ff4826b55380d74a48285a/users/60aee4f7a30845f69db2e71f02fa139b/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:20:41.347527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7145-37cc-4481-a47f-3446184099a8 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:41.348116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7145-37cc-4481-a47f-3446184099a8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:41.348116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7145-37cc-4481-a47f-3446184099a8 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:41.349585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 641/1274] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:41.354761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:41.355166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:41.355325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:41.374076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:41.375250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:41.442889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 634/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:41.467018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a7145-37cc-4481-a47f-3446184099a8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:41.571443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a7145-37cc-4481-a47f-3446184099a8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:41.589164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:41.605408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:41.606111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:41.606401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:41.606685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:41.616555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 29 14:20:41.621470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:41.626231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:41.627193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 642/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Jul 29 14:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:41.629427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:41.710521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:41.728196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414621e-87a6-40f4-822b-56993cfb45f1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:41.761529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:41.762147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:41.762479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:41.762750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:41.763466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:41.763773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:41.765172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:41.773974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 635/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Jul 29 14:20:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:20:41.779856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:41.880092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe635d5-a5b0-4bc2-978e-7530983ca2f1 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:41.914394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:41.915023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:41.915303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:41.915578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:41.926102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:41.928730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:41.943799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:41.944933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:41.945901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:41.945901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:41.945901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:41.945901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:41.945901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 643/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:41.950832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:42.049688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f9f4-a3f6-4dcc-8a49-f473d44768c6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:42.083328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:42.083933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:42.084208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:42.084477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:42.085180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:42.085479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:42.086822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:42.098051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 636/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:42.104705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:42.135275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:42.203268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac0b9-2f85-45de-9071-ada57d84f456 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:42.237467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:42.238118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:42.238388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:42.238666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:42.246321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:42.247772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 29 14:20:42.258996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:42.260882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:42.262159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 644/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:42.266012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:42.269793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:42.363099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b063f-c10a-4e91-8ae1-22652dc691a6 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:42.396859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:42.397532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:42.397815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:42.398086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:42.398866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:42.399171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:42.400573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:42.417630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 637/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Jul 29 14:20:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:42.423436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:42.514394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:42.524502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda9803-f05e-42b2-b71d-44f63950ef5e None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:42.557567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:42.558179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:42.558463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:42.558803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:42.626749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:42.628141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:42.715371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:42.725481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 645/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Jul 29 14:20:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 29 14:20:42.730997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:42.793915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:42.833442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2993c23-0fb6-43fb-92f8-05656b467293 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:42.867611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:42.868242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:42.868520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:42.868790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:42.878513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:42.880915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 29 14:20:42.896749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:42.897858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:42.899133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 29 14:20:42.899133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 29 14:20:42.899133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 29 14:20:42.899133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:42.899133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 638/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:42.903679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:43.013677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35981-8ca1-4216-a09c-c9e6746a280a None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:43.047412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:43.048018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:43.048295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:43.048567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:43.049275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:43.049573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:43.050924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:43.068295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 646/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Jul 29 14:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:20:43.075065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:43.202951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43ae91-2a87-48a0-8184-51a3a3aef952 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:43.219802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:43.243094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:43.243744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:43.244023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:43.244294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:43.251293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:43.252737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 29 14:20:43.265328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:43.266416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: ERROR keystone.server.flask.application Jul 29 14:20:43.267451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 639/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Jul 29 14:20:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:43.272580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:43.328813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:43.383374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeabfd2-5494-45b6-be74-28696b0a2e2b None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:43.422344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:43.422870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:43.423148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:43.423420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:43.424165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:43.424686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:43.427397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30703e-406f-4deb-9afc-6e47333b974b None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:43.439667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 647/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Jul 29 14:20:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:43.446092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:43.560124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd811-2ce5-4781-b925-4e85ccaa4113 None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:43.599428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:43.599963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:43.600243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:43.600519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:43.601336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] RBAC: Adding query filter params (role.id=744756d2b40c45a2aa4ea4c4d207d3f1, scope.project.id=77d5d61ad1ff4826b55380d74a48285a, user.id=60aee4f7a30845f69db2e71f02fa139b) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:43.601647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:43.603228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ad680-6bc2-450c-aee0-58bf9c2ac98e None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:43.612548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 640/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Jul 29 14:20:43 2022] GET /identity/v3/role_assignments?user.id=60aee4f7a30845f69db2e71f02fa139b&scope.project.id=77d5d61ad1ff4826b55380d74a48285a&role.id=744756d2b40c45a2aa4ea4c4d207d3f1 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:45.651834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516c179-b672-4ca1-b2ab-9b5634a9bb0b None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:45.652395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516c179-b672-4ca1-b2ab-9b5634a9bb0b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:45.652395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516c179-b672-4ca1-b2ab-9b5634a9bb0b None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:45.653769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 648/1288] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:45.658723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:45.659133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:45.659273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:45.678689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:45.679865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:45.709100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:45.744922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 641/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:45.766058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516c179-b672-4ca1-b2ab-9b5634a9bb0b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:45.868255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516c179-b672-4ca1-b2ab-9b5634a9bb0b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:45.878863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:45.902675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:45.903247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:45.903523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:45.903794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:45.904414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:45.905827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:45.915199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:45.917591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:45.918649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 649/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:20:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:45.922519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:46.019831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443c69c-5b31-4763-b68a-5c654e0583e7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:46.053308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:46.053919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:46.054196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:46.054466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:46.055134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:46.055440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:46.056851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8e494-e2d8-49e2-88a9-1b0b6759d565 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:46.061667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 642/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 14:20:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:46.068670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:46.173220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fe660-42fe-4fc2-8f08-a430b44fb283 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:46.206771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:46.207410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:46.208350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:46.208723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] PATH_INFO: `/v3/services` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:46.209415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:46.209716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:46.211129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08248-c1e4-4dc8-894b-766b4c8dd227 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:46.215480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 650/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 14:20:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:48.219639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2bd5-f72a-4804-bf5c-04e5972743d2 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:48.219639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2bd5-f72a-4804-bf5c-04e5972743d2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:48.219639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2bd5-f72a-4804-bf5c-04e5972743d2 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:48.219639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 643/1293] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:48.220412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e47d-8ca6-4cf6-86a5-1bc7c7ea8e1e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:48.220412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e47d-8ca6-4cf6-86a5-1bc7c7ea8e1e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:48.220412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e47d-8ca6-4cf6-86a5-1bc7c7ea8e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:48.224621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3a10e47d-8ca6-4cf6-86a5-1bc7c7ea8e1e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:48.225871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3a10e47d-8ca6-4cf6-86a5-1bc7c7ea8e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:48.255844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:48.293115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 651/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:48.316344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c2bd5-f72a-4804-bf5c-04e5972743d2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:48.347946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:48.417933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c2bd5-f72a-4804-bf5c-04e5972743d2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:48.451407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:48.452009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:48.452289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:48.452563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:48.453280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:48.454638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae4fe4-78be-4ffc-a3db-e8ef712d853d None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:48.534979 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 644/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Jul 29 14:20:48 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:20:50.517660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04215c-97bc-4844-9a07-3900fb3fe6ef None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:50.518188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04215c-97bc-4844-9a07-3900fb3fe6ef None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:50.518293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04215c-97bc-4844-9a07-3900fb3fe6ef None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:50.519609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 652/1296] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:50.524866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:50.530228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:50.530228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:50.543689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:50.544869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:50.617476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 645/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:50.639333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04215c-97bc-4844-9a07-3900fb3fe6ef None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:50.742874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04215c-97bc-4844-9a07-3900fb3fe6ef None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:50.776567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:50.777157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:50.777438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:50.777709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:50.778326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:50.779692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:50.789194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:50.790535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:50.791474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 653/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:20:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:50.796601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:50.815908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:50.894352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da174b-77fb-4ab8-a802-b88474129e2a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:50.928097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:50.928712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:50.929019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:50.929290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:50.929972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:50.930275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:50.931638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:50.936603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 646/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 14:20:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:50.943454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:50.978659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:51.047502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beae0e-d91f-4b97-a85c-8d95a6c01711 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:51.081183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:51.081802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:51.082083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:51.082357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:51.083669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] RBAC: Adding query filter params (interface=public, service_id=6954890d1cda441f92ab810daf6eee67, region_id=RegionOne) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:51.083979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:51.085401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6bc0f-27ca-430b-a5bb-b490c4a19e14 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:51.090385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 654/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Jul 29 14:20:50 2022] GET /identity/v3/endpoints?service_id=6954890d1cda441f92ab810daf6eee67&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:51.096374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:51.180178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:51.198036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b16f52-01a9-4e65-a56d-834aae965c2a None admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:51.232438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:51.233076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:51.233358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:51.233631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:51.234288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:51.235645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c256491-5151-46dd-aa9a-c65ec96d5fdd None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:51.242405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 647/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Jul 29 14:20:51 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:20:53.225355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae780-e98e-41d8-9518-483a028c9656 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:53.225895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae780-e98e-41d8-9518-483a028c9656 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:53.225895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae780-e98e-41d8-9518-483a028c9656 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:53.227287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 655/1302] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:20:53.232573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:53.233013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:53.234025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:53.251884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:20:53.253100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:20:53.319843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 648/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:20:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 29 14:20:53.342099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae780-e98e-41d8-9518-483a028c9656 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:53.435936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:53.443362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae780-e98e-41d8-9518-483a028c9656 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:53.475426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:53.476054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:53.476331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:53.476601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:53.477245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:53.478598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 29 14:20:53.487170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 29 14:20:53.488383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.get_or_create( Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application with Lock( Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self._enter() Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application return self.creator() Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application created_value = creator( Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: ERROR keystone.server.flask.application Jul 29 14:20:53.490274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 656/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Jul 29 14:20:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 29 14:20:53.495187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:53.575388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:20:53.597361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdfff8-6075-4977-943c-202a7f597098 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:53.631229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:53.631839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:53.632119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:53.632395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] PATH_INFO: `/v3/services` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:53.633104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:20:53.633410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:53.634777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3be2f6-5ad7-4d5a-90c9-9c281e704ce4 None admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:53.639923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 649/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Jul 29 14:20:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:20:53.645957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:20:53.747431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60b99-6204-470f-a6f9-1713386b3c24 None admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:20:53.780491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:20:53.781130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:20:53.781411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:20:53.781685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:20:53.782403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:20:53.783759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1893bd2-3e5f-4a85-9ecd-30e3c452c55d None admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:20:53.889184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 657/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:20:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:21:03.678672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=30 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:21:03.885258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:26:59.656349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978784a2-e99f-45b8-b791-97ba5be7c583 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:26:59.657592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978784a2-e99f-45b8-b791-97ba5be7c583 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:26:59.658094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978784a2-e99f-45b8-b791-97ba5be7c583 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:26:59.663048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 650/1307] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:26:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:26:59.667844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a5b9-0e81-4930-8165-0579bd8f89b5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:26:59.668278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a5b9-0e81-4930-8165-0579bd8f89b5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:26:59.668448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a5b9-0e81-4930-8165-0579bd8f89b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:26:59.771611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-5a40a5b9-0e81-4930-8165-0579bd8f89b5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:26:59.773202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5a40a5b9-0e81-4930-8165-0579bd8f89b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:26:59.921140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 658/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:26:59.968884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:26:59.969355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:26:59.969697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:26:59.971422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 651/1309] 10.222.0.49 () {54 vars in 965 bytes} [Fri Jul 29 14:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:26:59.978515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755fcc-4149-4893-9301-3fd013b4c7e1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:26:59.978808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755fcc-4149-4893-9301-3fd013b4c7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:26:59.979089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755fcc-4149-4893-9301-3fd013b4c7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:00.074605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c8755fcc-4149-4893-9301-3fd013b4c7e1 None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:00.076168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c8755fcc-4149-4893-9301-3fd013b4c7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:00.098910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:00.165146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 659/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Jul 29 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:00.173221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:00.174963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:00.294642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c194fc6-6dd3-424e-bf9d-1ee4013f9921 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:00.372435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:00.373084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:00.373361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:00.374539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:00.376171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:00.431619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:00.489675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:00.491492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8df01f-98d7-4841-9ff9-4b8cc3ea8d81 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:00.512022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 652/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:02.863300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce31cda-f644-43df-852f-bd2429348b15 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:02.863875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce31cda-f644-43df-852f-bd2429348b15 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:02.863875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce31cda-f644-43df-852f-bd2429348b15 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:02.865442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 660/1312] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:02.870486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5851d-e625-41e4-a3df-5074866ca5a3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:02.870905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5851d-e625-41e4-a3df-5074866ca5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:02.871089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5851d-e625-41e4-a3df-5074866ca5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:02.956230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6ba5851d-e625-41e4-a3df-5074866ca5a3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:02.957495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6ba5851d-e625-41e4-a3df-5074866ca5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:03.064084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 653/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:03.117740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e382-c727-473e-b46d-3be68269453c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:03.118098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e382-c727-473e-b46d-3be68269453c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:03.118488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e382-c727-473e-b46d-3be68269453c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:03.120376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 661/1314] 10.222.0.49 () {54 vars in 965 bytes} [Fri Jul 29 14:27:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:03.127965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdcef4-8157-443e-b011-7d182e6327bc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:03.128419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdcef4-8157-443e-b011-7d182e6327bc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:03.128711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdcef4-8157-443e-b011-7d182e6327bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:03.143072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:03.228549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-45bdcef4-8157-443e-b011-7d182e6327bc None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:03.229830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-45bdcef4-8157-443e-b011-7d182e6327bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:03.314937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 654/1315] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Jul 29 14:27:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:03.323687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6e382-c727-473e-b46d-3be68269453c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:03.325346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-56f6e382-c727-473e-b46d-3be68269453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:03.442171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6e382-c727-473e-b46d-3be68269453c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:03.518341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:03.521427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:03.522198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:03.522548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:03.522888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:03.524786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:03.633572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:03.635004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238eb2b-195b-47a1-978d-43df94282939 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:03.652373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 662/1316] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:27:06.398380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:06.398911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:06.399030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:06.400310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 655/1317] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:06.406585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cfb08-703b-4c84-802c-5108c5d163a0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:06.406992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cfb08-703b-4c84-802c-5108c5d163a0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:06.407083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cfb08-703b-4c84-802c-5108c5d163a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:06.473615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-487cfb08-703b-4c84-802c-5108c5d163a0 None None] MFA Rules not processed for user `2a01c135c7dc48f58bf2bfb169ad38a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:06.474808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-487cfb08-703b-4c84-802c-5108c5d163a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:06.528471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:06.573545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 663/1318] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:06.613560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:06.615073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:06.718038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524c4b1-b8d5-4d47-b1c1-a807204471e4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:06.720701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:06.796647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:06.797275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:06.797552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:06.797821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:06.799414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:06.908847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:06.910312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b7012-93d6-420c-b112-a6a772808982 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:06.929200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 656/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:09.795684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613360-7589-4091-b9db-874e72368754 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:09.796446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613360-7589-4091-b9db-874e72368754 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:09.796732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613360-7589-4091-b9db-874e72368754 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:09.798138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 664/1320] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:09.803171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a3923-5003-4d3f-b38a-536b7ee05ff1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:09.803577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a3923-5003-4d3f-b38a-536b7ee05ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:09.803729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a3923-5003-4d3f-b38a-536b7ee05ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:09.845608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:09.869466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-323a3923-5003-4d3f-b38a-536b7ee05ff1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:09.870683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-323a3923-5003-4d3f-b38a-536b7ee05ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:09.968378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 657/1321] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:10.009610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3613360-7589-4091-b9db-874e72368754 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:10.011116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b3613360-7589-4091-b9db-874e72368754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:10.113984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3613360-7589-4091-b9db-874e72368754 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:10.189107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:10.189726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:10.193956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:10.193956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:10.193956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:10.199430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:10.299301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:10.300726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65faba76-8066-4f1d-b26a-69ae48d971dc service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:10.319054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 665/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:13.395494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:13.396085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:13.396085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:13.397472 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 658/1323] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:13.402715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f42f7-95bd-431d-b978-cc066797128d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:13.403164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f42f7-95bd-431d-b978-cc066797128d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:13.403313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f42f7-95bd-431d-b978-cc066797128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:13.469580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2b0f42f7-95bd-431d-b978-cc066797128d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:13.470713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2b0f42f7-95bd-431d-b978-cc066797128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:13.535057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:13.578123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 666/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:13.615311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:13.616854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:13.728052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79505-342d-4092-a0f7-d77707b754ba None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:13.741168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:13.805313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:13.805938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:13.806223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:13.806495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:13.808106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:13.926757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:13.928125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bb859-a8bd-496f-a540-4696cb975203 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:13.955348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 659/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:16.020397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:16.020948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:16.020948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:16.022388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 667/1326] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:16.027743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1134a-2a84-4a88-bc77-735cd78f4292 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:16.028173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1134a-2a84-4a88-bc77-735cd78f4292 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:16.028456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1134a-2a84-4a88-bc77-735cd78f4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:16.076375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:16.095820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2fb1134a-2a84-4a88-bc77-735cd78f4292 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:16.097074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2fb1134a-2a84-4a88-bc77-735cd78f4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:16.200152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 660/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:27:16.237699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:16.239322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:16.346706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3447c-3933-449d-87f8-7c2e066b3b64 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:16.426925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:16.427548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:16.427828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:16.428100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:16.430003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:16.447446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:16.542753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:16.544200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336b60d-e7a7-4cb5-8aac-543936b13c3a service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:16.570755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 668/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:18.663084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:18.663656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:18.663762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:18.665222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 661/1329] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:18.670593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f814-8dde-4448-a9f7-da323de8d6d5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:18.671567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f814-8dde-4448-a9f7-da323de8d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:18.671790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f814-8dde-4448-a9f7-da323de8d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:18.741826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2e73f814-8dde-4448-a9f7-da323de8d6d5 None None] MFA Rules not processed for user `2a01c135c7dc48f58bf2bfb169ad38a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:18.743039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2e73f814-8dde-4448-a9f7-da323de8d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:18.812307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:18.845391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 669/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:18.884389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:18.885956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:18.990407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9643b-f364-4de7-9ae1-2e5815551f02 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:19.010309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:19.066591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:19.067211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:19.067494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:19.067767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:19.069414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:19.184603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:19.189208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089badd-1d37-4c75-ab7c-dd659d858ebd service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:19.223932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 662/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:21.692761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:21.693432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:21.693432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:21.694773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 670/1332] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:21.700096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dd31-48bb-4e3f-9b0c-9032096db35d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:21.700519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dd31-48bb-4e3f-9b0c-9032096db35d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:21.700663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dd31-48bb-4e3f-9b0c-9032096db35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:21.763327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:21.766679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5e79dd31-48bb-4e3f-9b0c-9032096db35d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:21.767893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5e79dd31-48bb-4e3f-9b0c-9032096db35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:21.862454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 663/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:21.898739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:21.900257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:22.004468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f59b-9ea6-4419-b280-0b72f41a5f5e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:22.078206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:22.078897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:22.079180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:22.079454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:22.081107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:22.108031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:22.190790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:22.192298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6ecc-6e77-4e8b-9966-225f6f900c33 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:22.210378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 671/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:24.236025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:24.236552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:24.236552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:24.238000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 664/1335] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:27:24.243211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c41ee-2ced-4b85-9245-d87d6eb641a5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:24.243622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c41ee-2ced-4b85-9245-d87d6eb641a5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:24.243759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c41ee-2ced-4b85-9245-d87d6eb641a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:24.310843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-348c41ee-2ced-4b85-9245-d87d6eb641a5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:24.312041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-348c41ee-2ced-4b85-9245-d87d6eb641a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:24.385668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:24.408883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 672/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:24.448319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:24.449914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:24.554006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291945-85d5-43ef-8fc0-0522bb8f42b6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:24.582647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:24.627817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:24.628428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:24.628705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:24.628977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:24.630606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:24.739576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:24.740921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecd49-d21d-4c7e-8d6f-1e4ac3259efa service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:24.759847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 665/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:27.426368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:27.426897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:27.426897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:27.428215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 673/1338] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:27.433917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fac7-f0c8-4b3b-9ef4-5cc9e4fc36d4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:27.434319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fac7-f0c8-4b3b-9ef4-5cc9e4fc36d4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:27.434469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8fac7-f0c8-4b3b-9ef4-5cc9e4fc36d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:27.499475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6de8fac7-f0c8-4b3b-9ef4-5cc9e4fc36d4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:27.501375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6de8fac7-f0c8-4b3b-9ef4-5cc9e4fc36d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:27.509400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:27.595932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 666/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:27.631811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:27.633338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:27.737266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7910de-d26c-48c1-999b-95c5cba1c3d0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:27.809602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:27.810214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:27.810497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:27.810775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:27.812933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:27.847974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:27.920678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:27.922688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061180c-c3a3-479b-8714-1c81f0129434 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:27.942033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 674/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:32.523602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:32.524041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:32.524211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:32.525649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 667/1341] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:32.530985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c286-caeb-49e8-b7f5-048e42b86daa None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:32.532012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c286-caeb-49e8-b7f5-048e42b86daa None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:32.532303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c286-caeb-49e8-b7f5-048e42b86daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:32.599216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b7b0c286-caeb-49e8-b7f5-048e42b86daa None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:32.600428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b7b0c286-caeb-49e8-b7f5-048e42b86daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:32.689976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:32.701336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 675/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:32.740684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:32.742341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:32.849844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366be227-33d5-4359-bfae-8fc3bead0098 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:32.887241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:32.924219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:32.924831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:32.925144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:32.925416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:32.927012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:33.037513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:33.038944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d24d01-6054-474c-9259-2e1896f24bc3 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:33.056790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 668/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:27:36.076948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:36.077503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:36.077613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:36.079846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 676/1344] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:36.084847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087bcb-05fb-498f-b12c-45e4c15c66fb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:36.085264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087bcb-05fb-498f-b12c-45e4c15c66fb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:36.085444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087bcb-05fb-498f-b12c-45e4c15c66fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:36.152308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a2087bcb-05fb-498f-b12c-45e4c15c66fb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:36.153506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a2087bcb-05fb-498f-b12c-45e4c15c66fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:36.176297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:36.256533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 669/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:36.294236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:36.295819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:36.402635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319290c-aba1-4baf-938d-01f61085d5f6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:36.482356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:36.482979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:36.483261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:36.483533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:36.485880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:36.534391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:36.604087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:36.605581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0b1b5-dc7b-4338-ab31-cc325a4823f4 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:36.625282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 677/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:27:40.920353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51065da-a24c-459e-b895-e604d147443c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:40.921244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51065da-a24c-459e-b895-e604d147443c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:40.922334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51065da-a24c-459e-b895-e604d147443c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:40.923754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 670/1347] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:40.929637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301deb2-0fd1-4708-afe3-12996eaa08e8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:40.930100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301deb2-0fd1-4708-afe3-12996eaa08e8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:40.930209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301deb2-0fd1-4708-afe3-12996eaa08e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:41.005813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6301deb2-0fd1-4708-afe3-12996eaa08e8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:41.007044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6301deb2-0fd1-4708-afe3-12996eaa08e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:41.110177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:41.112675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 678/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:41.151499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51065da-a24c-459e-b895-e604d147443c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:41.153114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e51065da-a24c-459e-b895-e604d147443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:41.266168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51065da-a24c-459e-b895-e604d147443c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:41.316832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:41.346453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:41.347110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:41.347341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:41.348148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:41.350368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:41.463223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:41.464516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3a4e7-9f99-4d05-b5a7-dfbdd3c90735 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:41.483174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 671/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:27:46.341396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d0a25-fb88-4a78-ae41-b8c1982c6473 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:46.341897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d0a25-fb88-4a78-ae41-b8c1982c6473 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:46.342192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d0a25-fb88-4a78-ae41-b8c1982c6473 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:46.344012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 679/1350] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:46.351040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b58f-1872-42cb-b8b5-a64c07496b0d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:46.351040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b58f-1872-42cb-b8b5-a64c07496b0d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:46.353063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3b58f-1872-42cb-b8b5-a64c07496b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:46.502305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-eae3b58f-1872-42cb-b8b5-a64c07496b0d None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:46.503514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eae3b58f-1872-42cb-b8b5-a64c07496b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:46.550253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:46.669358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 672/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:27:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:27:51.040100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be6e8-c79c-483f-bb4a-71cf7df3c7fa None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:51.040661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be6e8-c79c-483f-bb4a-71cf7df3c7fa None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:51.040661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be6e8-c79c-483f-bb4a-71cf7df3c7fa None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:51.042015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 680/1352] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:51.047229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7229f-bc6c-4d19-96bb-2fe3343ad1bf None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:51.047649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7229f-bc6c-4d19-96bb-2fe3343ad1bf None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:51.047837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7229f-bc6c-4d19-96bb-2fe3343ad1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:51.126076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-eed7229f-bc6c-4d19-96bb-2fe3343ad1bf None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:51.127275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eed7229f-bc6c-4d19-96bb-2fe3343ad1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:51.213511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:51.253830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 673/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:52.399579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd904e-f31b-469e-9a49-cf513c8c6425 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:52.400112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd904e-f31b-469e-9a49-cf513c8c6425 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:52.400112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd904e-f31b-469e-9a49-cf513c8c6425 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:52.401501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 681/1354] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:52.407326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55baa3-bf1b-43c4-99ca-319dfd149f94 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:52.407793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55baa3-bf1b-43c4-99ca-319dfd149f94 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:52.408001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55baa3-bf1b-43c4-99ca-319dfd149f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:52.485929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8d55baa3-bf1b-43c4-99ca-319dfd149f94 None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:52.487034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8d55baa3-bf1b-43c4-99ca-319dfd149f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:52.592567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 674/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:27:56.163152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e6e8-e01c-4191-a256-d27cca885f70 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:56.163639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e6e8-e01c-4191-a256-d27cca885f70 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:56.163734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e6e8-e01c-4191-a256-d27cca885f70 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:56.165524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 682/1356] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:56.170519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f662e-ca1d-4677-b23b-df8ccb4eefcc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:56.172203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f662e-ca1d-4677-b23b-df8ccb4eefcc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:56.172783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f662e-ca1d-4677-b23b-df8ccb4eefcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:56.212712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:56.247792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-193f662e-ca1d-4677-b23b-df8ccb4eefcc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:56.249187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-193f662e-ca1d-4677-b23b-df8ccb4eefcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:56.351896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 675/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:27:56.406542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:56.407053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:56.407564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:56.409398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 683/1358] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 14:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:56.418599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130148-e49f-46f4-8a97-095077f21b02 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:56.419761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130148-e49f-46f4-8a97-095077f21b02 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:56.420221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130148-e49f-46f4-8a97-095077f21b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:56.506728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-af130148-e49f-46f4-8a97-095077f21b02 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:56.508278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-af130148-e49f-46f4-8a97-095077f21b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:56.566706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:56.641209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 676/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:27:56.649377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:56.651029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:56.820015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130824-9c18-494f-90bf-b8b5a11ebe34 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:56.899711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:56.944565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:56.945222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:56.945790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:56.946074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:56.947863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:57.067150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:27:57.069051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337e3b0-8e3a-4b44-b770-a055f9065a8d service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:27:57.091588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 684/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:27:59.337726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:59.338273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:59.338273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:59.339705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 677/1361] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:27:59.344818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309812d-802a-4e83-95a9-ebbd6d7d34d3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:59.345411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309812d-802a-4e83-95a9-ebbd6d7d34d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:59.345692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309812d-802a-4e83-95a9-ebbd6d7d34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:59.415536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f309812d-802a-4e83-95a9-ebbd6d7d34d3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:27:59.416768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f309812d-802a-4e83-95a9-ebbd6d7d34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:59.456849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:59.521092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 685/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:27:59.568079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:27:59.569798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:27:59.747152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcda4b-b948-4e82-9511-02505a0ec5dd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:27:59.757602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:27:59.906850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:27:59.907642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:27:59.907927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:27:59.908197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:27:59.909874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:00.009889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:00.011556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ff92-3764-4e73-b203-4eab536384a7 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:00.032187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 678/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:02.461239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39fdf2-6023-46ea-9a02-98d8d14bde7c None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:02.461771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39fdf2-6023-46ea-9a02-98d8d14bde7c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:02.461771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39fdf2-6023-46ea-9a02-98d8d14bde7c None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:02.463058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 686/1364] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:02.469948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d0154-2b6f-458f-ac30-21484c17902a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:02.470412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d0154-2b6f-458f-ac30-21484c17902a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:02.470494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d0154-2b6f-458f-ac30-21484c17902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:02.486586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:02.540230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ed3d0154-2b6f-458f-ac30-21484c17902a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:02.541445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ed3d0154-2b6f-458f-ac30-21484c17902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:02.645322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 679/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:02.691541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:02.691948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:02.692495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:02.694036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 687/1366] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 14:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:02.699334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95301f-4d79-4d6b-82f8-9fa09fd81a08 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:02.699626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95301f-4d79-4d6b-82f8-9fa09fd81a08 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:02.699901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95301f-4d79-4d6b-82f8-9fa09fd81a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:02.784028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7f95301f-4d79-4d6b-82f8-9fa09fd81a08 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:02.785452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7f95301f-4d79-4d6b-82f8-9fa09fd81a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:02.823506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:02.915937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 680/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:02.924663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:02.926285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:03.088641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd821fc9-8f88-4c45-9ef1-929f0def64d1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:03.110647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:03.220392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:03.221041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:03.221330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:03.221607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:03.223240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:03.320119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:03.321778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354a48-4987-4ca5-b7a9-252cf807386b service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:03.341437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 688/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:04.272969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51916d13-7b73-4368-a79e-840dfb078124 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:04.273553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51916d13-7b73-4368-a79e-840dfb078124 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:04.273553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51916d13-7b73-4368-a79e-840dfb078124 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:04.274995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 681/1369] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:04.280264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762a77-a09b-4133-988d-949b87f833da None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:04.280678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762a77-a09b-4133-988d-949b87f833da None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:04.281546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762a77-a09b-4133-988d-949b87f833da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:04.315618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:04.352506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-cc762a77-a09b-4133-988d-949b87f833da None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:04.354138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cc762a77-a09b-4133-988d-949b87f833da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:04.461116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 689/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:04.809684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:04.809684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:04.809684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:04.812937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 682/1371] 10.222.0.49 () {54 vars in 964 bytes} [Fri Jul 29 14:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:04.816835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602ba24-7a6e-4757-90c3-a1a068f65b56 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:04.817172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602ba24-7a6e-4757-90c3-a1a068f65b56 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:04.817453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602ba24-7a6e-4757-90c3-a1a068f65b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:04.899534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d602ba24-7a6e-4757-90c3-a1a068f65b56 None None] MFA Rules not processed for user `c7c0f85f0b514af0bc2de51aeaf4ffdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:04.900745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d602ba24-7a6e-4757-90c3-a1a068f65b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:04.951333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:04.981726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 690/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Jul 29 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:04.989888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:04.991383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:05.107012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5b3dd-7618-437e-9f49-a190487d75f9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:05.168125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:05.193524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:05.194147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:05.194448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:05.194703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:05.196351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:05.317485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:05.318949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a47e9-52a4-49c7-ac0e-14f7b5951aa0 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:05.339155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 683/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:05.539574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:05.540080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:05.540583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:05.542176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 691/1374] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:05.547305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0cb00-5b32-442b-bc90-599654144c55 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:05.547723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0cb00-5b32-442b-bc90-599654144c55 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:05.547895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0cb00-5b32-442b-bc90-599654144c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:05.614499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4ba0cb00-5b32-442b-bc90-599654144c55 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:05.615593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4ba0cb00-5b32-442b-bc90-599654144c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:05.639126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:05.727314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 684/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:05.772091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:05.773681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:05.920720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c228ef7-3b1b-49bf-beb7-f4a3559fd7d0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:05.973514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:06.037082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:06.037708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:06.037987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:06.038264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:06.039881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:06.131753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:06.133379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefbdce-9401-47a7-a1cf-6c20cb6d0da9 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:06.152876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 692/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:08.326599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:08.327106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:08.327232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:08.329212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 685/1377] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:28:08.334510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563cea-cfdc-46be-8bcc-cb3a44a676b4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:08.334954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563cea-cfdc-46be-8bcc-cb3a44a676b4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:08.335080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563cea-cfdc-46be-8bcc-cb3a44a676b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:08.403766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a4563cea-cfdc-46be-8bcc-cb3a44a676b4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:08.404984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a4563cea-cfdc-46be-8bcc-cb3a44a676b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:08.412464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:08.531994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 693/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:08.577415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:08.579002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:08.724560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c27492-092d-43d9-bfaa-371a4ce5f194 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:08.735400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:08.860162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:08.860559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:08.860840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:08.861141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:08.862858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:08.991526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:08.993092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40c79f-47ff-421c-921d-74bde7171ad0 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:09.018724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 686/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:11.200237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:11.200760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:11.200760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:11.202344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 694/1380] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:11.208106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39233fb-b2ac-4a50-8b95-9b2e17ac3a52 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:11.208517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39233fb-b2ac-4a50-8b95-9b2e17ac3a52 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:11.208707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39233fb-b2ac-4a50-8b95-9b2e17ac3a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:11.220232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:11.276831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d39233fb-b2ac-4a50-8b95-9b2e17ac3a52 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:11.278148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d39233fb-b2ac-4a50-8b95-9b2e17ac3a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:11.384770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 687/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:11.430165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:11.431781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:11.577340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:11.588145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aab78e-8527-4f2e-9276-90315a3d410b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:11.708900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:11.709454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:11.709732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:11.710005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:11.711636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:11.813968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:11.815645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51207c-4b8a-4dca-a71d-86e344dff60e service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:11.832909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:11.839538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 695/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:14.007888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:14.008398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:14.008398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:14.009972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 688/1383] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:14.015328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d390dd6-6c6e-4ca7-8062-fd63d84d7ee8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:14.015777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d390dd6-6c6e-4ca7-8062-fd63d84d7ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:14.015982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d390dd6-6c6e-4ca7-8062-fd63d84d7ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:14.091745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6d390dd6-6c6e-4ca7-8062-fd63d84d7ee8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:14.092968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6d390dd6-6c6e-4ca7-8062-fd63d84d7ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:14.215660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 696/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:14.260425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:14.262459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:14.381482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:14.453562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f275bc-3b9c-4d04-a6e3-bc0244d95e3e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:14.587517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:14.588141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:14.588418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:14.588687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:14.590426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:14.642478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:14.683437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:14.685224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c5e84-1007-49c0-8c89-8086a0164077 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:14.704580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 689/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:16.721304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:16.721919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:16.721919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:16.723234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 697/1386] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:16.728721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2f49-ae17-4f49-9e66-9113944668e2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:16.729338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2f49-ae17-4f49-9e66-9113944668e2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:16.729338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2f49-ae17-4f49-9e66-9113944668e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:16.797962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-565e2f49-ae17-4f49-9e66-9113944668e2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:16.799174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-565e2f49-ae17-4f49-9e66-9113944668e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:16.903379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 690/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:28:16.954868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:16.956701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:17.038933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:17.112648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d61ec-0023-450e-977f-f643fb28e523 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:17.233157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:17.233852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:17.234293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:17.235199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:17.236804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:17.277309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:17.333557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:17.335061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d82e3e-fc76-4d2e-ba7c-32a0389438f8 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:17.354398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 698/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:19.529808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:19.530371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:19.530371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:19.532252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 691/1389] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:19.537410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e73-1695-4274-b01e-3a7162e38682 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:19.537852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e73-1695-4274-b01e-3a7162e38682 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:19.537963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35311e73-1695-4274-b01e-3a7162e38682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:19.604667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-35311e73-1695-4274-b01e-3a7162e38682 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:19.605927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-35311e73-1695-4274-b01e-3a7162e38682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:19.704626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 699/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:19.749027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:19.750611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:19.782244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:19.899975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97675b4-cfe8-4449-98b4-b9649f4a5aad None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:20.001865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:20.013384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:20.015651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:20.015651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:20.015651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:20.016686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:20.105813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:20.107332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fac9-f486-49fa-8ab3-906819e87389 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:20.125477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 692/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:20.782575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5ed7-5e42-43da-916b-45a797e1fb50 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:20.783143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5ed7-5e42-43da-916b-45a797e1fb50 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:20.783143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c5ed7-5e42-43da-916b-45a797e1fb50 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:20.784479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 700/1392] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:28:20.789791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b990d-7b29-4e5c-abbc-f03954a90134 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:20.790245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b990d-7b29-4e5c-abbc-f03954a90134 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:20.790345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b990d-7b29-4e5c-abbc-f03954a90134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:20.857360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ed5b990d-7b29-4e5c-abbc-f03954a90134 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:20.858563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ed5b990d-7b29-4e5c-abbc-f03954a90134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:20.916430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:20.955688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 693/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:21.011395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:21.011753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:21.012136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:21.013630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 701/1394] 10.222.0.49 () {54 vars in 966 bytes} [Fri Jul 29 14:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:21.019479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1751db-594a-45cd-937e-6be979375457 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:21.019767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1751db-594a-45cd-937e-6be979375457 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:21.020040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1751db-594a-45cd-937e-6be979375457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:21.095953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4d1751db-594a-45cd-937e-6be979375457 None None] MFA Rules not processed for user `a86fe5edd283422cb3d5650845927693`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:21.097191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4d1751db-594a-45cd-937e-6be979375457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:21.175788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 694/1395] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Jul 29 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:21.183893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:21.185537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:21.199657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:21.292536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5220-0d2b-4b71-8bdd-6535aa758765 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:21.371307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:21.371853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:21.372136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:21.372414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:21.374178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:21.444511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:21.486536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:21.487971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2b97-91bb-4bfa-9f9d-80f844308b15 service glance] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:21.506366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 702/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:22.293014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbe481-105b-4606-87d0-ad0680cb860f None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.293556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbe481-105b-4606-87d0-ad0680cb860f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.293654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbe481-105b-4606-87d0-ad0680cb860f None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.294959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 695/1397] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:22.300843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.301667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.301953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.350797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae12a0-84d2-4de1-895f-a16e699e7577 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.351093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae12a0-84d2-4de1-895f-a16e699e7577 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.351375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae12a0-84d2-4de1-895f-a16e699e7577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.375232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:22.376455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.419140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2bae12a0-84d2-4de1-895f-a16e699e7577 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:22.420356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2bae12a0-84d2-4de1-895f-a16e699e7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.428708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:22.476378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 703/1398] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:28:22.509307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 696/1399] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:28:22.522067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:22.523284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.552297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:22.558957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6ca0-cfe2-4619-b343-3e332a6632cb None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.559241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6ca0-cfe2-4619-b343-3e332a6632cb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.559521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6ca0-cfe2-4619-b343-3e332a6632cb None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.561340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 697/1400] 10.222.0.49 () {54 vars in 943 bytes} [Fri Jul 29 14:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:28:22.568005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.568287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.568565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.644218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932b932-ff8d-45c2-abf4-605683a02315 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:22.645387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] MFA Rules not processed for user `3189acc363fe42339aedae0a604c8b42`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:22.646606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.731230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:22.755897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 698/1401] 10.222.0.49 () {58 vars in 1040 bytes} [Fri Jul 29 14:28:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:28:22.761450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:22.762257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.762457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.762674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.764606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.765603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:22.775187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fbfa2-959c-4a6c-b13f-7e8b15c19566 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:22.782982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:22.859842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:22.861489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb04a6-bf60-4f41-abd9-cee84b958a62 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:22.880125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 704/1402] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:22.882759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:22.883378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:22.883684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:22.883955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:22.885977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:22.974067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:22.975661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fef87e-053f-4c60-813b-8cbd1252da43 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:22.984463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 699/1403] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 14:28:25.041471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:25.042133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:25.042239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:25.045038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 705/1404] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:25.049813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4700b4-52b1-483f-a1e6-95904e25a56d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:25.050277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4700b4-52b1-483f-a1e6-95904e25a56d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:25.050388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4700b4-52b1-483f-a1e6-95904e25a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:25.063701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:25.130000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7d4700b4-52b1-483f-a1e6-95904e25a56d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:25.133221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7d4700b4-52b1-483f-a1e6-95904e25a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:25.259229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 700/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:25.311149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:25.313453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:25.482249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:25.484847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f686-5ed7-4f45-b21c-b45a7f1aeada None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:25.615271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:25.616209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:25.616483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:25.616753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:25.618387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:25.721581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:25.723426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1968e-09e6-44cd-9963-746209c6bb8b service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:25.744113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 706/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:27.931134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:27.931676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:27.931676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:27.933317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 701/1407] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:27.938983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034ce0-0df6-49a8-ad09-0fe8df8903ba None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:27.939425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034ce0-0df6-49a8-ad09-0fe8df8903ba None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:27.939523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034ce0-0df6-49a8-ad09-0fe8df8903ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:27.944025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:28.013901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-35034ce0-0df6-49a8-ad09-0fe8df8903ba None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:28.015127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-35034ce0-0df6-49a8-ad09-0fe8df8903ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:28.129885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 707/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:28.175109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:28.176623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:28.274580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:28.342671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8156e2f-c08f-4fd6-974f-74d1378ce3b6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:28.467148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:28.467885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:28.468250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:28.468572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:28.470331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:28.526356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:28.569349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:28.570306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d857ec3-6932-4681-b3eb-fcf8879d53a1 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:28.590992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 702/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:38.127378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=64 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:38.588231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=23 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:46.524731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dbc0-e53c-4fcf-9966-c6d9f2e4e131 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:46.525315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dbc0-e53c-4fcf-9966-c6d9f2e4e131 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:46.525315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dbc0-e53c-4fcf-9966-c6d9f2e4e131 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:46.526728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 708/1410] 10.222.0.49 () {54 vars in 918 bytes} [Fri Jul 29 14:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:46.531898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392c66-fe97-41c6-aa90-0332f825486d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:46.532314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392c66-fe97-41c6-aa90-0332f825486d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:46.532484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66392c66-fe97-41c6-aa90-0332f825486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:46.607984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-66392c66-fe97-41c6-aa90-0332f825486d None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:46.609779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-66392c66-fe97-41c6-aa90-0332f825486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:46.716352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 703/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jul 29 14:28:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:28:46.730780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993efc8-a77c-4d81-a4e3-34806fa4ef99 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:46.731063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993efc8-a77c-4d81-a4e3-34806fa4ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:46.731342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993efc8-a77c-4d81-a4e3-34806fa4ef99 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:46.732687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 709/1412] 10.222.0.49 () {54 vars in 918 bytes} [Fri Jul 29 14:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:46.737865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371bafe-921b-4730-9780-d5e9e57aaded None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:46.738183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371bafe-921b-4730-9780-d5e9e57aaded None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:46.738458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371bafe-921b-4730-9780-d5e9e57aaded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:46.809774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:46.813754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1371bafe-921b-4730-9780-d5e9e57aaded None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:46.814973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1371bafe-921b-4730-9780-d5e9e57aaded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:46.934605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 704/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jul 29 14:28:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:49.514278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:49.514874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:49.514984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:49.516335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:49.523080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbe2f-3a98-41fc-8839-9851ba48f03a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:49.523523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbe2f-3a98-41fc-8839-9851ba48f03a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:49.523670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbe2f-3a98-41fc-8839-9851ba48f03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:49.599631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a31fbe2f-3a98-41fc-8839-9851ba48f03a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:49.600968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a31fbe2f-3a98-41fc-8839-9851ba48f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:49.658762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:49.702102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:49.747274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:49.748848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:49.930062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358fa28-a6c8-483d-832c-cfee55b3a393 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:50.003646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:50.044803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:50.045456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:50.045734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:50.046004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:50.047630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:50.137582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:50.139182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e61cc-3157-4adb-b74b-ae343952596d service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:50.158492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 711/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:51.056053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:51.056620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:51.056620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:51.057985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:51.063153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4f2f-4584-4dc0-9e20-0d09ae85b906 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:51.063602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4f2f-4584-4dc0-9e20-0d09ae85b906 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:51.063709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4f2f-4584-4dc0-9e20-0d09ae85b906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:51.140932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e3de4f2f-4584-4dc0-9e20-0d09ae85b906 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:51.142197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e3de4f2f-4584-4dc0-9e20-0d09ae85b906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:51.171214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:51.242823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:51.392369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:51.393889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:51.537880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff356-eba4-46de-9b6e-7bde37a043e6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:51.578185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:51.651227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:51.651760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:51.652037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:51.652310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:51.653955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:51.740544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:51.742164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ac3ba-5eb8-48cb-9cbe-0991cf93dbc9 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:51.760340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 707/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:52.642204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:52.642752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:52.642752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:52.644110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:52.648929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742c476-0b1f-40d6-aac4-0191085136e2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:52.649383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742c476-0b1f-40d6-aac4-0191085136e2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:52.649511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742c476-0b1f-40d6-aac4-0191085136e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:52.703151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:52.724372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4742c476-0b1f-40d6-aac4-0191085136e2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:52.725593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4742c476-0b1f-40d6-aac4-0191085136e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:52.820115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:52.975303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:52.976811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:53.123032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac159b-5dbe-4d60-b116-458c6e66b291 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:53.135668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:53.238063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:53.238605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:53.238885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:53.239158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:53.240796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:53.330544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:53.332155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68810e14-1301-4131-b9ea-844a7f5bd5b0 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:53.349650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 714/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:54.204655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:54.205191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:54.205288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:54.206661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:54.211679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed04b8-c058-4499-9833-bdc412ef3934 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:54.212103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed04b8-c058-4499-9833-bdc412ef3934 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:54.212231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed04b8-c058-4499-9833-bdc412ef3934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:54.232581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:54.289759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8eed04b8-c058-4499-9833-bdc412ef3934 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:54.290968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8eed04b8-c058-4499-9833-bdc412ef3934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:54.387160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:54.537040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:54.538818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:54.649597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:54.686536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a48015-ab57-4a81-984c-f4464aa370b9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:54.797866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:54.798400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:54.798677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:54.798946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:54.800534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:54.875887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:54.888904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:54.891230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c79e3-2386-4211-9f80-a10288dc172d service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:54.909222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 710/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:55.778894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:55.779519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:55.779519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:55.781602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:28:55.786525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009652-2a9a-4352-96eb-8c2c12d41e05 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:55.786957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009652-2a9a-4352-96eb-8c2c12d41e05 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:55.787079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009652-2a9a-4352-96eb-8c2c12d41e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:55.862093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-45009652-2a9a-4352-96eb-8c2c12d41e05 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:55.863213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-45009652-2a9a-4352-96eb-8c2c12d41e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:55.957978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:56.106523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:56.108066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:56.193091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:56.255078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05355713-4d0d-4321-9569-d7fdffafc37a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:56.372487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:56.373137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:56.373424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:56.373711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:56.375794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:56.453788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:56.468612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:56.470324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8901-5352-43dc-9978-59198bad9179 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:56.489889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 717/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:57.418626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:57.419180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:57.419180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:57.420442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:57.425847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c553fc-38f9-4bb7-a7af-dfae004f2d71 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:57.426301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c553fc-38f9-4bb7-a7af-dfae004f2d71 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:57.426429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c553fc-38f9-4bb7-a7af-dfae004f2d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:57.502519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f4c553fc-38f9-4bb7-a7af-dfae004f2d71 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:57.503831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f4c553fc-38f9-4bb7-a7af-dfae004f2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:57.600047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:57.750756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:57.752250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:57.800692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:57.896408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319a0b8-86b2-415e-b272-88f8954cf431 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:58.010282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:58.010821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:58.011099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:58.011374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:58.013013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:58.020381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:58.102305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:58.103924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3e1e9-88ab-452d-8f71-1e7296149d40 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:58.122814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 713/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:28:59.003129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:59.003740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:59.003740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:59.005074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:28:59.010075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e908c-7d83-45a3-a059-18ee2b98e0ee None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:59.010521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e908c-7d83-45a3-a059-18ee2b98e0ee None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:59.010630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e908c-7d83-45a3-a059-18ee2b98e0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:59.086857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7c3e908c-7d83-45a3-a059-18ee2b98e0ee None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:28:59.088062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7c3e908c-7d83-45a3-a059-18ee2b98e0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:59.158153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:59.188481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:28:59.339095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:28:59.340702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:59.359451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:59.489541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7e79-898e-4f00-9ecf-26b1756defc1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:28:59.576588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:28:59.603716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:28:59.604291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:28:59.604564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:28:59.604851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:28:59.606514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:28:59.695695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:28:59.697336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e1ff6-16b6-4b5f-b80e-d66750d62122 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:28:59.715468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 720/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:00.596686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:00.597292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:00.597292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:00.598647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:00.603568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8049b1-70dd-4d68-908c-a5cb5c3a6d68 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:00.604000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8049b1-70dd-4d68-908c-a5cb5c3a6d68 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:00.604104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8049b1-70dd-4d68-908c-a5cb5c3a6d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:00.680834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3a8049b1-70dd-4d68-908c-a5cb5c3a6d68 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:00.682075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3a8049b1-70dd-4d68-908c-a5cb5c3a6d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:00.723527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:00.777985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:00.927558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:00.929280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:01.073012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08fe28-e391-4cd3-86fd-d743416da76d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:01.126126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:01.185381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:01.185935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:01.186217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:01.186527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:01.188743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:01.277072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:01.278671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bc142-11f8-4a77-82c1-8b1e854eb44a service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:01.297181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 716/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:02.153624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:02.154166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:02.154166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:02.158782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:02.160623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd12c4-77a7-45f2-9be8-0b83f3ddb793 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:02.161110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd12c4-77a7-45f2-9be8-0b83f3ddb793 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:02.161217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd12c4-77a7-45f2-9be8-0b83f3ddb793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:02.238972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-58cd12c4-77a7-45f2-9be8-0b83f3ddb793 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:02.240428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-58cd12c4-77a7-45f2-9be8-0b83f3ddb793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:02.242399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:02.338262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:29:02.489434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:02.490930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:02.636929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d0237-f0e2-4e6f-a7df-e8a6ce10fd84 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:02.663822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:02.748478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:02.749635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:02.749922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:02.750216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:02.751805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:02.840625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:02.842255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ff58-ca48-48d7-9246-c5c571e6888f service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:02.861325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 723/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:04.734980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:04.735545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:04.735545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:04.736861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:04.742170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d176-b76d-4b5a-a471-887a9ec015cc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:04.742619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d176-b76d-4b5a-a471-887a9ec015cc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:04.742718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7d176-b76d-4b5a-a471-887a9ec015cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:04.783461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:04.809497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-07d7d176-b76d-4b5a-a471-887a9ec015cc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:04.812343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-07d7d176-b76d-4b5a-a471-887a9ec015cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:04.907537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:04.946728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:04.948258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:05.052130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3e8a1-559b-4676-b011-0430d73d544a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:05.083603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:05.125995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:05.126607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:05.126886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:05.127159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:05.128764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:05.239166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:05.240600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39280-13a7-4087-9635-55fc198bf0f8 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:05.258630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 719/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:07.409855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:07.410440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:07.410440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:07.411854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:07.417276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f35f2c-bc28-413f-bff4-460e64a8d82d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:07.417712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f35f2c-bc28-413f-bff4-460e64a8d82d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:07.417843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f35f2c-bc28-413f-bff4-460e64a8d82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:07.484045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-55f35f2c-bc28-413f-bff4-460e64a8d82d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:07.485290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-55f35f2c-bc28-413f-bff4-460e64a8d82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:07.497786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:07.587055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:07.626573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:07.628083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:07.731927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325067a-06f4-4426-8e66-e5a2002110e0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:07.744188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:07.806050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:07.806667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:07.806949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:07.807223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:07.808828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:07.916830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:07.918286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd005a-82af-4d5c-b3f5-ae4259aa6a4b service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:07.937915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 726/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:10.302485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:10.303036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:10.303036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:10.304649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:10.309960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1846-7952-41a7-a26e-d4098f080b35 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:10.310374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1846-7952-41a7-a26e-d4098f080b35 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:10.310513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1846-7952-41a7-a26e-d4098f080b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:10.358135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:10.376152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7dde1846-7952-41a7-a26e-d4098f080b35 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:10.377396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7dde1846-7952-41a7-a26e-d4098f080b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:10.473346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:10.512530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:10.514115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:10.616891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816cb76-3f5b-4842-a476-a3cfc31194dd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:10.659392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:10.693573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:10.694185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:10.694466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:10.694738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:10.696362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:10.924285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:10.926371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c5e6a-99cb-4d7c-9d8b-f90369015f60 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:10.944617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 722/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:13.076574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:13.077153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:13.077280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:13.078702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:13.083805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f7d0e-949b-4c5b-9840-561a1420bffc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:13.084230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f7d0e-949b-4c5b-9840-561a1420bffc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:13.084352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f7d0e-949b-4c5b-9840-561a1420bffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:13.148962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b90f7d0e-949b-4c5b-9840-561a1420bffc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:13.150219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b90f7d0e-949b-4c5b-9840-561a1420bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:13.171905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:13.245933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:13.285810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:13.287343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:13.390199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be74f-a0a7-4564-9d2d-82755fa78f47 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:13.410104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:13.462691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:13.463301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:13.463579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:13.463849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:13.465480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:13.574100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:13.575534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269f3e-8c23-422a-a213-d4801e8b1985 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:13.594883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 729/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:15.637382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:15.637894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:15.637999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:15.639349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:15.644611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d693ba-d60c-4231-8eda-ed59bcfc97d6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:15.645071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d693ba-d60c-4231-8eda-ed59bcfc97d6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:15.645191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d693ba-d60c-4231-8eda-ed59bcfc97d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:15.710271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:15.712981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-73d693ba-d60c-4231-8eda-ed59bcfc97d6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:15.714253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-73d693ba-d60c-4231-8eda-ed59bcfc97d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:15.811213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:15.849178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:15.850740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:15.955199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2cce5-85e7-4f94-b9d8-1218fd68f364 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:16.004594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:16.028706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:16.029352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:16.029634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:16.029906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:16.032112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:16.138700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:16.140789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1a1ed-2d16-4aff-98eb-a0fd1113bd0e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:16.160338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 725/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:18.199792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:18.200325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:18.200325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:18.201753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:18.207996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177eb2-89a4-462a-ba45-1d275a509f9e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:18.208411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177eb2-89a4-462a-ba45-1d275a509f9e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:18.208580 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177eb2-89a4-462a-ba45-1d275a509f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:18.275233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-35177eb2-89a4-462a-ba45-1d275a509f9e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:18.276423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-35177eb2-89a4-462a-ba45-1d275a509f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:18.306177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:18.372083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:18.411004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:18.412486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:18.516950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd93e8-5cd7-48a0-ad7a-d8ac34e4214a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:18.667405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:18.714508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:18.715114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:18.715390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:18.715660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:18.717418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:18.826345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:18.828802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8292c9-2c15-4433-ab5e-dc0731915460 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:18.846621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 732/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:20.898244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:20.898819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:20.898819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:20.900164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:20.905615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bfa3c-449d-4432-8ff6-9f4d44b4abc7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:20.906010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bfa3c-449d-4432-8ff6-9f4d44b4abc7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:20.906010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bfa3c-449d-4432-8ff6-9f4d44b4abc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:20.971153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3b8bfa3c-449d-4432-8ff6-9f4d44b4abc7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:20.972345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3b8bfa3c-449d-4432-8ff6-9f4d44b4abc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:20.980541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:21.067810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:21.105557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:21.107074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:21.211229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c5183-06ef-4835-b751-f620cb7f960a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:21.264932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:21.282620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:21.283224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:21.283495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:21.283762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:21.285386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:21.391677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:21.393105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b378bfa-ef30-4b71-bc86-f8fde6bf2269 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:21.410950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 728/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:23.444002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:23.444539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:23.444539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:23.445859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:23.451291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef988a86-88e9-4d3e-8c61-4548ac2686cb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:23.451766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef988a86-88e9-4d3e-8c61-4548ac2686cb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:23.451866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef988a86-88e9-4d3e-8c61-4548ac2686cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:23.516137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ef988a86-88e9-4d3e-8c61-4548ac2686cb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:23.517368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ef988a86-88e9-4d3e-8c61-4548ac2686cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:23.555304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:23.616833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:23.654409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:23.656243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:23.759743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc87cf-77b2-4de7-86f6-d1be365768ed None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:23.796967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:23.833924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:23.834529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:23.834808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:23.835071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:23.836655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:23.943221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:23.944642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc685d-6d6b-4625-b8ed-42f6a298d209 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:23.962872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 735/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:26.023693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:26.024240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:26.024240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:26.025623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:26.031346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0b6d-6cd5-4793-9880-3b9a6282d1c8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:26.031805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0b6d-6cd5-4793-9880-3b9a6282d1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:26.031913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0b6d-6cd5-4793-9880-3b9a6282d1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:26.096490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-89ab0b6d-6cd5-4793-9880-3b9a6282d1c8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:26.097711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-89ab0b6d-6cd5-4793-9880-3b9a6282d1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:26.115994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:26.195079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:26.232380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:26.234714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:26.338092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bdad9-c494-486a-9f2c-1b8478996f4c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:26.403962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:26.411347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:26.413068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:26.413068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:26.413068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:26.414320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:26.523586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:26.525038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de746575-7889-412d-a941-f1f959975b64 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:26.545170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 731/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:28.574858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:28.575463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:28.575463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:28.576773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:28.582446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded379-a982-4f41-b13c-ace56f9f1ea0 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:28.582903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded379-a982-4f41-b13c-ace56f9f1ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:28.583012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded379-a982-4f41-b13c-ace56f9f1ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:28.647827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-27ded379-a982-4f41-b13c-ace56f9f1ea0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:28.649097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-27ded379-a982-4f41-b13c-ace56f9f1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:28.697618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:28.745761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:28.783321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:28.784874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:28.887978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470f2f1-1e2f-47a8-ac2a-fe721d07341e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:28.935661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:28.963411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:28.964204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:28.964694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:28.965276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:28.967683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:29.084651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:29.086120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8f4d9-4bda-4374-9a1f-30a1fddb8e23 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:29.104211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 738/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:38.743918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=22 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:39.101940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=66 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:48.669777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74d098-ebcc-4e65-a4fd-e88cee18e5b1 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:48.670344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74d098-ebcc-4e65-a4fd-e88cee18e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:48.670344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74d098-ebcc-4e65-a4fd-e88cee18e5b1 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:48.671952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 733/1471] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:48.677198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6442-c41b-44d6-8069-cd729483050a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:48.677666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6442-c41b-44d6-8069-cd729483050a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:48.677777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6442-c41b-44d6-8069-cd729483050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:48.743706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-37fd6442-c41b-44d6-8069-cd729483050a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:48.744906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-37fd6442-c41b-44d6-8069-cd729483050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:48.841843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 739/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:48.925870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:48.926154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:48.926427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:48.927786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 734/1473] 10.222.0.49 () {54 vars in 966 bytes} [Fri Jul 29 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:48.932954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129d9e-2d7e-4a61-bddf-7e54caec5ff6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:48.933267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129d9e-2d7e-4a61-bddf-7e54caec5ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:48.933539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129d9e-2d7e-4a61-bddf-7e54caec5ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:49.010655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c8129d9e-2d7e-4a61-bddf-7e54caec5ff6 None None] MFA Rules not processed for user `a86fe5edd283422cb3d5650845927693`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:49.011874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c8129d9e-2d7e-4a61-bddf-7e54caec5ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:49.032264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:49.087925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 740/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Jul 29 14:29:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:49.095791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:49.097325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:49.201189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de193b0-9643-413d-aedb-decfbe6397ab None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:49.273460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:49.274073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:49.274355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:49.274631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:49.276246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:49.329949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:49.384463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:49.385934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2a582-eef1-47ed-bd66-a17194f2843b service glance] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:49.404130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 735/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:51.524917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:51.525548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:51.525548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:51.526963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 741/1476] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:51.533318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7b176-9d2c-4601-855d-e58d0bdf043a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:51.534068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7b176-9d2c-4601-855d-e58d0bdf043a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:51.534068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7b176-9d2c-4601-855d-e58d0bdf043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:51.613087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-20b7b176-9d2c-4601-855d-e58d0bdf043a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:51.614300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-20b7b176-9d2c-4601-855d-e58d0bdf043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:51.724333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 736/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:29:51.748182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:51.749863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:51.877913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12135b-dce6-434b-b3bc-5de066482d75 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:51.927080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:51.971828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:51.972474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:51.972753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:51.973050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:51.973749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:51.975481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:29:51.975481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:29:51.975481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:29:51.976607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af91fd-149e-4899-a924-696c3a80b4a8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:51.996413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 742/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Jul 29 14:29:51 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:29:54.040287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:54.040821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:54.040821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:54.042258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 737/1479] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:54.047340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ee8e-2220-42f8-b27f-821d9e1e4a7b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:54.047748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ee8e-2220-42f8-b27f-821d9e1e4a7b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:54.047897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ee8e-2220-42f8-b27f-821d9e1e4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:54.112645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-cc08ee8e-2220-42f8-b27f-821d9e1e4a7b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:54.113871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cc08ee8e-2220-42f8-b27f-821d9e1e4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:54.214497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 743/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:54.258555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:54.260041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:54.270154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:54.403771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681af648-b0bb-4222-8ca7-2dc3e9dac415 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:54.483038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:54.515543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:54.516158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:54.516625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:54.517150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:54.519102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:54.607178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:54.609869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a1db-2216-4817-be45-f1c989ccfbb9 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:54.629091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 738/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:56.809720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:56.810265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:56.810265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:56.812242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 744/1482] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:56.817654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41494-a1e7-4268-8774-5d08aa191d69 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:56.818063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41494-a1e7-4268-8774-5d08aa191d69 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:56.818195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41494-a1e7-4268-8774-5d08aa191d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:56.884310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e8b41494-a1e7-4268-8774-5d08aa191d69 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:56.885545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e8b41494-a1e7-4268-8774-5d08aa191d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:56.930743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:56.980844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 739/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:29:57.018474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:57.020103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:57.043769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:57.124248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354cb64-f0c9-4f6c-a04b-92f6c8afd892 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:57.198129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:57.198748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:57.199028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:57.199301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:57.200906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:57.274781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:57.309629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:57.311108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7f46-967a-4780-b882-671dfc646248 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:57.329759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 745/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:29:59.382316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:59.382928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:59.382928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:59.384228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 740/1485] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:29:59.390099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a89c-097b-4fb5-90c3-ccb7059213d2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:59.390547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a89c-097b-4fb5-90c3-ccb7059213d2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:59.390652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a89c-097b-4fb5-90c3-ccb7059213d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:59.456559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0f64a89c-097b-4fb5-90c3-ccb7059213d2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:29:59.457787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0f64a89c-097b-4fb5-90c3-ccb7059213d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:59.552008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 746/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:29:59.590798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:29:59.592297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:59.693760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd192f1e-9664-4b2a-964c-0632c4b322d0 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:29:59.763650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:29:59.766173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:29:59.766794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:29:59.767078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:29:59.767357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:29:59.769002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:29:59.880531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:29:59.882669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21766794-bd61-416a-82fb-287902de3702 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:29:59.909662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 741/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:30:02.085984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:02.086541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:02.086541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:02.087962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 747/1488] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:30:02.093327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aa7e6-cc41-48e9-9c4e-d49789a7b50c None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:02.093775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aa7e6-cc41-48e9-9c4e-d49789a7b50c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:02.093873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68aa7e6-cc41-48e9-9c4e-d49789a7b50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:02.158836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a68aa7e6-cc41-48e9-9c4e-d49789a7b50c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:30:02.160054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a68aa7e6-cc41-48e9-9c4e-d49789a7b50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:02.212668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:30:02.257856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 742/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:30:02.298499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:30:02.300031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:02.336810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:30:02.404331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b727a08-b034-44fb-bbf1-0459be73c08b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:30:02.478950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:30:02.479490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:02.479770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:02.480040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:02.481682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:02.560682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:30:02.590243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:30:02.591662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637380ad-b43f-4c30-a240-7c5e2431fb28 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:30:02.610638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 748/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:30:04.996650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:04.997250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:04.997250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:04.998560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 743/1491] 10.222.0.49 () {54 vars in 926 bytes} [Fri Jul 29 14:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:30:05.003910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33a30-1e39-4d7d-ae36-dbfb8421d464 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:05.004365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33a30-1e39-4d7d-ae36-dbfb8421d464 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:05.004479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad33a30-1e39-4d7d-ae36-dbfb8421d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:05.070451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fad33a30-1e39-4d7d-ae36-dbfb8421d464 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:30:05.071661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fad33a30-1e39-4d7d-ae36-dbfb8421d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:05.168021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 749/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Fri Jul 29 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:30:05.207406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:30:05.208947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:05.313319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273162-8034-49ed-b398-92365208fcd2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:30:05.385455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:30:05.386061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:30:05.386367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:30:05.386670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:30:05.388209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:30:05.397678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:30:05.497244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:30:05.498761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bef328-2059-4cec-a9e7-9b1be457000b service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:30:05.517280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 744/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:30:15.166186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=82 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:30:15.514767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=50 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:47.390042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:47.390636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:47.390636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:47.466833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:31:47.468034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:47.563373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 750/1494] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:31:47.571680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:47.572174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:47.572274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:47.647887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:31:47.649033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:47.743893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 745/1495] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:31:47.748556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:47.749788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:47.762068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2792-fb9b-498e-aab3-8df05dfb6790 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:47.823590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:47.856807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:47.857449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:47.857726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:47.857996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:47.858717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:31:47.859015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:47.860027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:47.860027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:31:47.860027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:31:47.860903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:47.872405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 751/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:31:47.881367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:47.890479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605336-f947-4172-9f03-3bf72be38f32 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:47.952050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:47.984958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:47.985603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:47.985879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:47.986148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:47.986991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:47.989427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:47.989427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:31:47.989427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:31:47.990916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:48.121408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 746/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:31:47 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:31:48.125941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:48.127604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:48.163024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf0e1-de29-472a-a036-06d9c5b7484a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:48.262676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:48.263303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:48.263586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:48.263861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:48.264520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:48.265569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:48.265569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:31:48.265569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:31:48.266771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:48.371241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 752/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:31:48 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:31:48.375481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:48.377016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:48.439628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942ab8-cbc0-4729-9898-99a2d44fb9cc admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:48.521365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:48.531404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:31:48.535721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:31:48.536419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:48.548515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 747/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:31:48.553298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:48.555068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:48.566917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d3b18-41c6-40ec-af12-0dbe68a1eafe admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:48.591697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:48.662848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:48.663463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:48.663746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:48.664022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:48.664749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:48.665805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:48.665805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:31:48.665805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:31:48.666689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e5d26-fcf1-4dfc-861d-348bce6b0da4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:48.679540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 753/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:31:48.684021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:48.685564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:48.718080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a57d66-29cb-476f-a1b6-7d78c8f5c847 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:48.813586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:48.814176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:48.814456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:48.814727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] PATH_INFO: `/v3/projects/ba94acc503e5489984cf9675612e004b/users/b5a00c9b0cd4484e9280259a3dfc3dea/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:48.837018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba94acc503e5489984cf9675612e004b, user_id=b5a00c9b0cd4484e9280259a3dfc3dea, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:48.838092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:48.838092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:31:48.838092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:31:48.840030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc334c-18f9-47f6-b8c9-be80678816bf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:48.939472 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 748/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:31:48 2022] PUT /identity/v3/projects/ba94acc503e5489984cf9675612e004b/users/b5a00c9b0cd4484e9280259a3dfc3dea/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:31:48.952140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d979-f7f8-4931-81dc-22e4e17076fb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:48.952431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d979-f7f8-4931-81dc-22e4e17076fb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:48.952709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d979-f7f8-4931-81dc-22e4e17076fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.009992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d762d979-f7f8-4931-81dc-22e4e17076fb None None] MFA Rules not processed for user `b5a00c9b0cd4484e9280259a3dfc3dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:31:49.011246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d762d979-f7f8-4931-81dc-22e4e17076fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:49.089079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 754/1502] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:31:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:31:49.102534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:49.102820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:49.103096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.104438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 749/1503] 10.222.0.49 () {54 vars in 964 bytes} [Fri Jul 29 14:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:31:49.109146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71312e-6cbc-449f-928a-e757efa8b8c0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:49.109431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71312e-6cbc-449f-928a-e757efa8b8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:49.109729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71312e-6cbc-449f-928a-e757efa8b8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.142800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:49.189506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-1c71312e-6cbc-449f-928a-e757efa8b8c0 None None] MFA Rules not processed for user `c7c0f85f0b514af0bc2de51aeaf4ffdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:31:49.190715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1c71312e-6cbc-449f-928a-e757efa8b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:49.266233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 755/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Jul 29 14:31:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:31:49.274068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:49.275570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:49.326010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:49.379496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ee503-8487-4968-aedb-0f1232832f15 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:49.451799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:49.452413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:49.452693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:49.452967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.454592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:49.551224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:49.552649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:49.556332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:49.570141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 750/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:31:49.615672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229377d-1bba-4787-97b2-4ceecbbb4f83 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:49.616096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229377d-1bba-4787-97b2-4ceecbbb4f83 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:49.616209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229377d-1bba-4787-97b2-4ceecbbb4f83 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.617523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 756/1506] 10.222.0.49 () {52 vars in 830 bytes} [Fri Jul 29 14:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:31:49.621235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:49.622433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:49.721379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaab47f-df9b-4211-8fd0-a592e33c183f service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:49.814983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:49.816201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:49.816476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:49.816829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] PATH_INFO: `/v3/users/b5a00c9b0cd4484e9280259a3dfc3dea` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:49.829486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a00c9b0cd4484e9280259a3dfc3dea)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:49.830537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:49.830537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:31:49.830537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:31:49.832146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:49.841679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:50.039534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a00c9b0cd4484e9280259a3dfc3dea'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:50.058292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a00c9b0cd4484e9280259a3dfc3dea'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:50.071859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a00c9b0cd4484e9280259a3dfc3dea'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:50.072413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a00c9b0cd4484e9280259a3dfc3dea'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:50.147697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a00c9b0cd4484e9280259a3dfc3dea'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:50.153590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 751/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:31:49 2022] DELETE /identity/v3/users/b5a00c9b0cd4484e9280259a3dfc3dea => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:31:50.158255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:50.163071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:50.163071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:50.215668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:31:50.216867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:50.293189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:50.305987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 757/1508] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:31:50.315842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:50.317103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:50.417471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817eef-5584-4d87-b6a6-b1d0517be3cc admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:50.471204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:31:50.490105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:50.490723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:50.491006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:50.491283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:50.492924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:50.603068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:50.604433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba46b2-da4f-4ab8-8227-2e224d901f73 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:50.622926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 752/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:31:51.155833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:31:51.157231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:31:51.169274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7473d3-c59e-4452-a88b-4f0c357b07d1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:31:51.268007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:31:51.268657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:31:51.268932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:31:51.269233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] PATH_INFO: `/v3/projects/ba94acc503e5489984cf9675612e004b` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:31:51.279894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba94acc503e5489984cf9675612e004b)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:31:51.280924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:31:51.280924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:31:51.280924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:31:51.282160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:31:51.408672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba94acc503e5489984cf9675612e004b', 'request_id': 'req-55f49e46-26d5-47e4-9194-6715a491e1d5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:51.495959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba94acc503e5489984cf9675612e004b', 'request_id': 'req-55f49e46-26d5-47e4-9194-6715a491e1d5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:51.509944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba94acc503e5489984cf9675612e004b', 'request_id': 'req-55f49e46-26d5-47e4-9194-6715a491e1d5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:51.533670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] The token cache is being invalidate because project ba94acc503e5489984cf9675612e004b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:31:51.533956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:51.534374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-55f49e46-26d5-47e4-9194-6715a491e1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:31:51.535772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 758/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:31:51 2022] DELETE /identity/v3/projects/ba94acc503e5489984cf9675612e004b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:32:00.620588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=62 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:32:01.531712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:33:31.637544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:31.638140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:31.638251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:31.639578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:31.644356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971a08-0664-44ee-aa44-72826dc8c9d3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:31.648784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971a08-0664-44ee-aa44-72826dc8c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:31.648784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971a08-0664-44ee-aa44-72826dc8c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:31.725851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-13971a08-0664-44ee-aa44-72826dc8c9d3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:31.727150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-13971a08-0664-44ee-aa44-72826dc8c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:31.830360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:31.992331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:31.994503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:32.113654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a9e59-0504-47f6-aeda-c4caac1fdb9d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:32.119769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:32.120385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:32.120669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:32.120944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:32.129031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:32.130610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65a7e8-57b8-486b-b348-66dd951bf292 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:32.139830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 754/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:32.972423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6eebd-21d7-4cf1-a4d8-5cf946888c08 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:32.973017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6eebd-21d7-4cf1-a4d8-5cf946888c08 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:32.973017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6eebd-21d7-4cf1-a4d8-5cf946888c08 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:32.974375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:32.979216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47150a-dc56-4463-994c-b09bd8b446a6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:32.979650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47150a-dc56-4463-994c-b09bd8b446a6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:32.979768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47150a-dc56-4463-994c-b09bd8b446a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:32.999457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ff47150a-dc56-4463-994c-b09bd8b446a6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:33.000656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ff47150a-dc56-4463-994c-b09bd8b446a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:33.009222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:33:33.161687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6eebd-21d7-4cf1-a4d8-5cf946888c08 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:33.176172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6eebd-21d7-4cf1-a4d8-5cf946888c08 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:33.182602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:33.183209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:33.183483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:33.183753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:33.191946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:33.193646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d643d-aee4-4230-8793-ff2099a71067 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:33.203760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 761/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:34.012402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dda4c-2b9e-4b89-813c-0cb655a499ec None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:34.012931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dda4c-2b9e-4b89-813c-0cb655a499ec None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:34.013083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dda4c-2b9e-4b89-813c-0cb655a499ec None None] PATH_INFO: `/v3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:34.014394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 756/1517] 10.222.0.71 () {54 vars in 925 bytes} [Fri Jul 29 14:33:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:33:34.022005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e182c70-0b1c-4dbc-810a-09aafa3d9e0f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:34.022419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e182c70-0b1c-4dbc-810a-09aafa3d9e0f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:34.022550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e182c70-0b1c-4dbc-810a-09aafa3d9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:34.084166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4e182c70-0b1c-4dbc-810a-09aafa3d9e0f None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:34.085402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4e182c70-0b1c-4dbc-810a-09aafa3d9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:34.102490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:33:34.156556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 762/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Jul 29 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:42.994694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=56 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:33:43.741373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1105d-1499-48d7-8ed4-cbab3c12b004 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:43.741582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1105d-1499-48d7-8ed4-cbab3c12b004 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:43.741864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1105d-1499-48d7-8ed4-cbab3c12b004 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:43.743282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 757/1519] 10.222.0.49 () {54 vars in 961 bytes} [Fri Jul 29 14:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:43.749230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95f71-eb4c-4473-97da-b200abe92ca6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:43.749639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95f71-eb4c-4473-97da-b200abe92ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:43.749787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95f71-eb4c-4473-97da-b200abe92ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:43.769379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-92d95f71-eb4c-4473-97da-b200abe92ca6 None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:43.770583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-92d95f71-eb4c-4473-97da-b200abe92ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:43.779238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 763/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Jul 29 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:33:43.786914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1105d-1499-48d7-8ed4-cbab3c12b004 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:43.801128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1105d-1499-48d7-8ed4-cbab3c12b004 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:43.806305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:43.806909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:43.807192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:43.807470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:43.816017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:43.817646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185c00b-6ae0-46b8-867c-e6f3dc728cdd service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:43.828091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 758/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:44.415751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3c54-04c2-454d-a292-84a59b66c17e None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:44.415966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3c54-04c2-454d-a292-84a59b66c17e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:44.416220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db3c54-04c2-454d-a292-84a59b66c17e None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:44.417630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:44.423052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036cb50-ad77-4faa-aba9-ba92c7c281a7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:44.423530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036cb50-ad77-4faa-aba9-ba92c7c281a7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:44.423646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036cb50-ad77-4faa-aba9-ba92c7c281a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:44.444079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2036cb50-ad77-4faa-aba9-ba92c7c281a7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:44.445293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2036cb50-ad77-4faa-aba9-ba92c7c281a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:44.453252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:44.608718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db3c54-04c2-454d-a292-84a59b66c17e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:44.627070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db3c54-04c2-454d-a292-84a59b66c17e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:44.632702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:44.633356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:44.633638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:44.633915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:44.643416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:44.645014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63d922-3f8a-4042-9806-ec5509b11ece service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:44.655154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 765/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:45.486681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530582-6bb4-4a07-9ab8-5a24529c4b1e None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:45.487217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530582-6bb4-4a07-9ab8-5a24529c4b1e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:45.487217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530582-6bb4-4a07-9ab8-5a24529c4b1e None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:45.488543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 29 14:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:45.493218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b7cbc-d117-4988-9a9f-8456d7868132 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:45.493645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b7cbc-d117-4988-9a9f-8456d7868132 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:45.493754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b7cbc-d117-4988-9a9f-8456d7868132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:45.514188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-784b7cbc-d117-4988-9a9f-8456d7868132 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:45.515374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-784b7cbc-d117-4988-9a9f-8456d7868132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:45.523102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 29 14:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:45.677316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530582-6bb4-4a07-9ab8-5a24529c4b1e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:45.692300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530582-6bb4-4a07-9ab8-5a24529c4b1e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:45.697710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:45.698318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:45.698596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:45.698867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:45.707596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:45.709209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b218f-c614-47a1-a870-cf3dc941c50a service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:45.718526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 761/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:49.468650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:49.469220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:49.469220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:49.470519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:33:49.475582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b486c-2c80-47f2-a0ad-b3484daa0f40 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:49.475996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b486c-2c80-47f2-a0ad-b3484daa0f40 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:49.476145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b486c-2c80-47f2-a0ad-b3484daa0f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:49.495861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3e5b486c-2c80-47f2-a0ad-b3484daa0f40 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:49.497094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3e5b486c-2c80-47f2-a0ad-b3484daa0f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:49.505235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:49.578097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:49.579790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:49.658719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85ad2-ba8a-4c6a-907e-5ad8cfbdad5a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:49.662273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:49.662877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:49.663156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:49.663425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:49.672796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:49.674334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ae061-19ea-4cb3-b6f4-8cee4f0bb61b service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:49.683882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 768/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:51.682839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d00b3-25aa-437f-86b9-ba02d15f9e24 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:51.683384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d00b3-25aa-437f-86b9-ba02d15f9e24 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:51.683384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d00b3-25aa-437f-86b9-ba02d15f9e24 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:51.684743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:51.690180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb6e-fa5a-4c44-8f9e-02830cb253d6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:51.690616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb6e-fa5a-4c44-8f9e-02830cb253d6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:51.690744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb6e-fa5a-4c44-8f9e-02830cb253d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:51.710127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fd0eeb6e-fa5a-4c44-8f9e-02830cb253d6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:51.711440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fd0eeb6e-fa5a-4c44-8f9e-02830cb253d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:51.720092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:51.756836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d00b3-25aa-437f-86b9-ba02d15f9e24 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:51.768000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d00b3-25aa-437f-86b9-ba02d15f9e24 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:51.771470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:51.772073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:51.772353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:51.772628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:51.781620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:51.783040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c903c05-218d-4c78-8720-32998e154450 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:51.792328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 764/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:55.313403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:55.313970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:55.313970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:55.315310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:55.320399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591f20-d839-49aa-9b09-fcfa6c8f59cc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:55.320796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591f20-d839-49aa-9b09-fcfa6c8f59cc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:55.320955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591f20-d839-49aa-9b09-fcfa6c8f59cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:55.340562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8b591f20-d839-49aa-9b09-fcfa6c8f59cc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:55.341920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8b591f20-d839-49aa-9b09-fcfa6c8f59cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:55.349684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:55.651734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:55.653449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:55.719629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:33:55.732579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10ff3b-50c7-48bc-bba6-8d410fe415b8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:55.736225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:55.736841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:55.737160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:55.737438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:55.746354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:55.747786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488b2cc-55ed-4ab1-87ba-331e627246a6 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:55.757666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 771/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:33:57.767672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483883da-bd87-4558-95fc-aa914914878d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:57.768126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483883da-bd87-4558-95fc-aa914914878d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:57.768242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483883da-bd87-4558-95fc-aa914914878d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:57.769609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 29 14:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:33:57.774873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652d83-a4e9-4206-9f4a-7c7eb14737db None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:57.775281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652d83-a4e9-4206-9f4a-7c7eb14737db None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:57.775451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652d83-a4e9-4206-9f4a-7c7eb14737db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:57.795571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7c652d83-a4e9-4206-9f4a-7c7eb14737db None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:33:57.796755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7c652d83-a4e9-4206-9f4a-7c7eb14737db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:57.804393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 29 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:33:58.108939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483883da-bd87-4558-95fc-aa914914878d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:33:58.114016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-483883da-bd87-4558-95fc-aa914914878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:33:58.124923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483883da-bd87-4558-95fc-aa914914878d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:33:58.128493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:33:58.129144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:33:58.129427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:33:58.129705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:33:58.138520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:33:58.139943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb9738-d8d7-496e-964e-230665cc611e service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:33:58.149259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 767/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:34:07.790226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=11 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:34:08.145114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=33 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:27.858042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:27.859158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:27.859656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:27.891363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:27.892708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:27.907237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 773/1540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:27.917829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:27.918492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:27.918492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:27.958624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:27.959894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:27.974655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 768/1541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:27.979550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:27.993534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476a5b-4532-4fa3-bf7d-3f36c6cbbb1c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:27.999132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:27.999795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.000087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.000364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.005516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.005964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.007713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.023565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 774/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.032541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.062125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4593710-acb5-44e2-aa64-95c33603e941 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.062125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.062125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.062522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.062783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.063345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.065918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35eb41-8de6-43b9-8423-4bd6e1ad99ec admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.181128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 769/1543] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:28.185633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.202681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a345717-a7e3-4eb2-b571-a55dbb959ed1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.285141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.285785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.286072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.286354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.287045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.289020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.399699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461c15-5a46-42f4-b2c8-9d72f48d3a5b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.399902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461c15-5a46-42f4-b2c8-9d72f48d3a5b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.400355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461c15-5a46-42f4-b2c8-9d72f48d3a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.413966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 775/1544] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:28.420796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.446037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b17b2-1ebf-4094-ad57-bf456c46b4f4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.457682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-30461c15-5a46-42f4-b2c8-9d72f48d3a5b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.458919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-30461c15-5a46-42f4-b2c8-9d72f48d3a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.461970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.462618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.462903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.463186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.465119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.466748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088777e-019c-4007-8e0b-b6e16b096138 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.483559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 770/1545] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:28.485138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cd597-7dbb-44e8-9303-d98c062698e7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.485426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cd597-7dbb-44e8-9303-d98c062698e7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.486246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cd597-7dbb-44e8-9303-d98c062698e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.500071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 776/1546] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:28.502139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a52f1-e5eb-42ba-928b-a9d9eeee3c2d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.502436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a52f1-e5eb-42ba-928b-a9d9eeee3c2d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.502715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a52f1-e5eb-42ba-928b-a9d9eeee3c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.522630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2d7cd597-7dbb-44e8-9303-d98c062698e7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.524008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2d7cd597-7dbb-44e8-9303-d98c062698e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.527769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c07a52f1-e5eb-42ba-928b-a9d9eeee3c2d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.528985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c07a52f1-e5eb-42ba-928b-a9d9eeee3c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.535587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 771/1547] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.536820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93ae0-9648-4b9e-8ea6-9a16e8bc0192 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.537141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93ae0-9648-4b9e-8ea6-9a16e8bc0192 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.537420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93ae0-9648-4b9e-8ea6-9a16e8bc0192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.540247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 777/1548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.540247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.540247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.540247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.558673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-abf93ae0-9648-4b9e-8ea6-9a16e8bc0192 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.559858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.559986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-abf93ae0-9648-4b9e-8ea6-9a16e8bc0192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.561052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.568214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 772/1549] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.569424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626b96-743f-4576-bb24-2449d28a8600 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.569714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626b96-743f-4576-bb24-2449d28a8600 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.569990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626b96-743f-4576-bb24-2449d28a8600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.576359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 778/1550] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.577427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.586992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6ce60-fa1d-4c39-89aa-acf14c5ffbd1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.589109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ec626b96-743f-4576-bb24-2449d28a8600 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.590297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ec626b96-743f-4576-bb24-2449d28a8600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.591674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.592288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.592569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.592844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.593574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.595190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c23a58-6747-4923-8b5f-fc62825dd372 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.598368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 773/1551] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.599750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317795-922e-4822-8f56-d6b2bc1a0167 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.600139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317795-922e-4822-8f56-d6b2bc1a0167 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.600516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317795-922e-4822-8f56-d6b2bc1a0167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.609960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 779/1552] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:28.609960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c323a1-eb6f-40cf-8cc6-bf93c8c2a3b0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.609960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c323a1-eb6f-40cf-8cc6-bf93c8c2a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.609960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c323a1-eb6f-40cf-8cc6-bf93c8c2a3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.620805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-02317795-922e-4822-8f56-d6b2bc1a0167 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.622054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-02317795-922e-4822-8f56-d6b2bc1a0167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.628686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-34c323a1-eb6f-40cf-8cc6-bf93c8c2a3b0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.629886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-34c323a1-eb6f-40cf-8cc6-bf93c8c2a3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.631190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 774/1553] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.632464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a65d-a076-40c5-a70a-89a2cdb0f8ac None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.632755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a65d-a076-40c5-a70a-89a2cdb0f8ac None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.633068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a65d-a076-40c5-a70a-89a2cdb0f8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.638229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 780/1554] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.646450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4d60-95b1-4cc9-9790-01f26d2cdfe8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.646739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4d60-95b1-4cc9-9790-01f26d2cdfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.647025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4d60-95b1-4cc9-9790-01f26d2cdfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.652884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d640a65d-a076-40c5-a70a-89a2cdb0f8ac None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.654060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d640a65d-a076-40c5-a70a-89a2cdb0f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.663171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 775/1555] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.664446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.664738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.665043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.667019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d1cd4d60-95b1-4cc9-9790-01f26d2cdfe8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.668222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d1cd4d60-95b1-4cc9-9790-01f26d2cdfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.677051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 781/1556] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.678270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.678559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.678839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.686660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.687833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.695842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 776/1557] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.696701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.698213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.699700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.706538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb46700-c03f-4c77-aaf1-2c41a6848754 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.708462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 782/1558] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.709277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.714579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.715192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.715474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.715755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.716454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.716761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.717810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:28.717810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:35:28.717810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:35:28.718612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a45f7-af1e-42c9-9a32-6ed97cc17e68 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.718731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3d562-2ea7-48a5-9504-79d5dfce822a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.724135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.724787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.725095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.725372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] PATH_INFO: `/v3/projects/fdbc49e630fc461a858eb04c2dd52a82/users/de769d721b8b49c6a5d7d391412da122/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.731639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 777/1559] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.732843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b6c4-4c54-4b98-874b-d2f0017af9a8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.733170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b6c4-4c54-4b98-874b-d2f0017af9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.733449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102b6c4-4c54-4b98-874b-d2f0017af9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.742020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbc49e630fc461a858eb04c2dd52a82, user_id=de769d721b8b49c6a5d7d391412da122, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.743064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:28.743064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:35:28.743064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:35:28.744967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.757405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9102b6c4-4c54-4b98-874b-d2f0017af9a8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.758609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9102b6c4-4c54-4b98-874b-d2f0017af9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.766534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 778/1560] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.768541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.768828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.769133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.787974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:28.789482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:28.797589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 779/1561] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:28.798505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.808171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d892a6-f633-48b9-b4a2-de2a9409a112 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.819375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 783/1562] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:28 2022] PUT /identity/v3/projects/fdbc49e630fc461a858eb04c2dd52a82/users/de769d721b8b49c6a5d7d391412da122/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:28.819375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.863583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:28.866651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.871920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.874766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96bfc-1746-416c-a37e-03fcc6a653da admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.880111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.880726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.881028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.881307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.882004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.882309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.890245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.890451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 780/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.890451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.897826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 784/1564] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.897826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.902002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933abf52-0c50-48ae-94c5-b956ff4234f1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.906817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.907383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.907821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.907936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d3ec-fd9a-43f0-a7e0-5318339d3676 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.908199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.909110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.909420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.911006 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.912902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.913545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.913875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.914143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.914825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.915125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.916697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.923591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 781/1565] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.929406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.938453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 785/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.938453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.942477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe225f-2791-45a4-b9a1-8d9433bdba11 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.942868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e16acd-2ef7-4da1-9191-a6bd1cb41542 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.947069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.947402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.947668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.947947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.948079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.948223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.948341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.948534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.948908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.949238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.949352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.950823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.951156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.963067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 786/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.970369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:28.975282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-3f4a-4c73-b889-24f5a1e996d2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:28.980577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:28.981209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.981492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.981766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:28.982456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:28.982791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:28.984380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0dfa1f-753f-4f64-99ba-746c94e40c3d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:28.997384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 787/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:28.998659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:28.999221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:28.999355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.020077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:29.021316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:29.028805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 788/1569] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:29.029578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.042138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5f9f-c94a-481b-84b7-342056c2bb5c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.044442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.045075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.045358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.045634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.046298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.047307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:29.047307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:35:29.047307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:35:29.049068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.066539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 782/1570] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:28 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.067500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.077354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:29.128044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1134e-dcdd-4d90-b832-e03995731346 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.132754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.133405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.133686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.133959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.134619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.136570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.154115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 789/1571] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.154115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.215659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ebdc6-2f22-4ae7-b28b-a13fca3aaad4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.218925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 783/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 403 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.219647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.223620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.224237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.224515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.224788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.225477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.227423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.282351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65b600-9278-4bbe-99f4-ad9ebd4540fc admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.287434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.288041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.288320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.288592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.289282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.291179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.297051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 790/1573] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.297051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.359407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe5f3a-1f36-402c-b4f3-02eb1feba740 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.365011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.365636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.365918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.366189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.366957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.368860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.374248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:29.379664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 784/1574] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.380602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.442434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511553ff-bf53-4aa7-b4c9-405f78d8cb63 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.446941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.447562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.447839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.448116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.448777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.450739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.456041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 791/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.458397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.458794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:29.523210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c732b3-59ef-4b81-903b-2ad0c059d3ec admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.526881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.527540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.527840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.528102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.542613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 785/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.543449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.549485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:29.591538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.592986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.603630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 792/1577] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:29.604406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.604635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d02c-9316-4b62-a6c9-b1ca1b3689b3 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.609740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.610365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.611238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.611684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.612342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.613407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:29.613407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:35:29.613407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:35:29.614620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.614988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d6b4c-7ec4-4323-94c2-b26056508d65 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.620233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.620958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.621332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.621678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.623053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.624546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.712114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 786/1578] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.713026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.723421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f7a7-163e-4934-96e7-1ca1097ea502 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.727976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.728566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.728848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.729173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.729826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.733350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.739040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 793/1579] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.739427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.751352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247e94-984d-437d-ba81-4d3abdada5fe admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.756036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.756680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.756967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.757274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.757934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.759856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.821422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 787/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.821422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.829042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca33e5a-cf08-4c98-81cd-264906db0214 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.834399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.835008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.835293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.835566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.837380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.838183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.846444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 794/1581] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.849970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.861282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d963c-95f7-4d46-aab4-8f344052ac71 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.866039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.866634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.866933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.867186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.867852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.869786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.967797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 795/1582] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.971661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.972110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 788/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:29.973778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:29.982972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b06ca-a9c0-4e91-9d66-f263e2faa036 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.984162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e720e-a5a2-44ee-a531-1bfe526c7525 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:29.988389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.989070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.989348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.989576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.990437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.990547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:29.991361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:29.991704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:29.992084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:29.992814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:29.992956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:29.994775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.012558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 789/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.012558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.024551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0595667-d0fc-4c61-8f17-35c61898a686 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.029614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.030227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.030934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.030934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.031534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.033826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.044473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 790/1585] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.045377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.058584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfb7b6-56b7-471c-a45e-4c9f2569338b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.063420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.064267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.064617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.064922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.065655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.067247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.078167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 796/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:29 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:30.080720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.084896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 791/1587] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.086026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.091958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605857-0e1b-4ca5-88d9-9299b5647a20 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.096354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f750a-5e82-49b8-a002-e1ef3d5671e8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.097329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.097943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.098227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.098497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.099225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.100813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.101375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.101981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.102287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.102568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.103262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.104840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.111990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 797/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.112869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.121039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 792/1589] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.121039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.123120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6edccc-9cd0-49dc-96e0-00995213a776 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.128320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.128939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.129259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.129426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e965ca4-f380-4322-a3e5-4f579b21fd92 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.129536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.130231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.131855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.134470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.144052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.144052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.144052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.144052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.144052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.145333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 798/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.146496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.152778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 793/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.152930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.156273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:30.161658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2783c-ec40-4cdd-b2dd-7d9cf36d9eaa admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.165161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54240f9f-3041-4aae-9c8a-1482ffc13914 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.168332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.172805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.172805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.172805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.173250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.173375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.173635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.173635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.173635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.173912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.174018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.174018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.184329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 799/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.185596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.190808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 794/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.191858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.201437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9724a-a7e2-4e21-a79c-4b0d18a23c98 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.202479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd41f9-d845-4dc9-aba4-8af8394cb8b2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.207985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.208342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.208608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.208883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.209193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.209893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.210057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.210545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.210958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.211480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.211989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.214866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.222401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 795/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.226750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.232445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 800/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.232445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.238252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e081c-58ac-4d73-8888-da52e3be9c41 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.238677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85919c88-76fa-4439-95df-ea179b0e82d5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.243644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.243880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.244266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.244623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.244748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.244917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.245153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.245246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.245667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.245924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.247269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.247486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.263294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 796/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.263294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.263622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 801/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.263622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.275067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f15a7e-9292-43a4-b1ab-75446a1b16e2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.275282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9307-3e66-4991-b474-853fed7b03d9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.280326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.281111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.281465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.281826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] PATH_INFO: `/v3/projects/82cd919860e249acb9dea7620506725b/users/f4ae3fb6a7954a24b88df4f35f55020c/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.282464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.283575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.283959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.284295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] PATH_INFO: `/v3/projects/c8e78f4f35e54ff9bc37e22705074bcf/users/78e1329dc558419dba83bd62ea7d69c1/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.300939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cd919860e249acb9dea7620506725b, user_id=f4ae3fb6a7954a24b88df4f35f55020c, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.302860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e78f4f35e54ff9bc37e22705074bcf, user_id=78e1329dc558419dba83bd62ea7d69c1, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.304026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.305727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.372421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 802/1598] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/82cd919860e249acb9dea7620506725b/users/f4ae3fb6a7954a24b88df4f35f55020c/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.373370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.405304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 797/1599] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/c8e78f4f35e54ff9bc37e22705074bcf/users/78e1329dc558419dba83bd62ea7d69c1/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.406220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.419191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:30.447342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca75a2-9b85-4f1f-aadd-2fc5b608c4ce admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.480208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9d7a-c99d-4047-b264-fe05e95a80cf admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.485496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.486758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.487150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.487532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] PATH_INFO: `/v3/projects/1776579056dd420d81368ec51e2ddb95/users/30432a068ad8486db8af635e66776f90/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.503740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1776579056dd420d81368ec51e2ddb95, user_id=30432a068ad8486db8af635e66776f90, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.506561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.511964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.512608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.512914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.513221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.513932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.515572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.527118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 803/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:30.529924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.540123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994a1a-e1b8-4ec0-ba55-c5e552006ac3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.544685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.545389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.545665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.545937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] PATH_INFO: `/v3/projects/3ee32ab01d4749ff8f2ae792045c2bcb/users/797d321bbca441ebb7ed7d8db3b09c38/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.561751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee32ab01d4749ff8f2ae792045c2bcb, user_id=797d321bbca441ebb7ed7d8db3b09c38, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.565215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.584202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 798/1601] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/1776579056dd420d81368ec51e2ddb95/users/30432a068ad8486db8af635e66776f90/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.585095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.648299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703efce-836f-4e97-97d2-829fff3e998f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.655204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 804/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/3ee32ab01d4749ff8f2ae792045c2bcb/users/797d321bbca441ebb7ed7d8db3b09c38/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.660644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.708250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:30.719761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.721068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.721357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.721638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] PATH_INFO: `/v3/projects/44c869dc06224bf49f25fcf7aeb0e5a7/users/f3a38dc21bc34164aea48065da64823f/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.736675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c869dc06224bf49f25fcf7aeb0e5a7, user_id=f3a38dc21bc34164aea48065da64823f, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.739390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5563ea-6c33-4c00-8f30-07906b5343ae admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.741655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d13cd5-dcdf-4238-9d37-5f319beb2c89 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.746332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.746998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.747280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.747559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] PATH_INFO: `/v3/projects/7d4c357515b04f71ba99c3f1cae80ddd/users/d28e4b47b1fb4f24bdbe2ef9c99a6b95/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.762474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4c357515b04f71ba99c3f1cae80ddd, user_id=d28e4b47b1fb4f24bdbe2ef9c99a6b95, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:30.765173 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39628e1b-14ab-4a61-8d3a-b6e9e57844e5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:30.823281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 799/1603] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/44c869dc06224bf49f25fcf7aeb0e5a7/users/f3a38dc21bc34164aea48065da64823f/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.824632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.824924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.825245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.843361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:30.844564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:30.857720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:30.886491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 805/1604] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/7d4c357515b04f71ba99c3f1cae80ddd/users/d28e4b47b1fb4f24bdbe2ef9c99a6b95/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:30.888476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d71a7-3ebb-444c-861c-9b98f0234f1f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.888772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d71a7-3ebb-444c-861c-9b98f0234f1f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.889149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d71a7-3ebb-444c-861c-9b98f0234f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:30.904606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 800/1605] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:30.905993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:30.919135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc53dd9-0166-4a40-8cf8-fef121b7eaf8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:30.930078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-076d71a7-3ebb-444c-861c-9b98f0234f1f None None] MFA Rules not processed for user `78e1329dc558419dba83bd62ea7d69c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:30.931434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-076d71a7-3ebb-444c-861c-9b98f0234f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:30.988504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:30.989969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:30.990326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:30.990683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] PATH_INFO: `/v3/projects/930127f777e34dd1911ed0443282d41a/users/f489c80ce3324b818096ede43b690fb4/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.003564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 806/1606] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.005131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a81ec1-4574-4e2d-b5d5-3ffaba7489bb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.005616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a81ec1-4574-4e2d-b5d5-3ffaba7489bb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.006048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a81ec1-4574-4e2d-b5d5-3ffaba7489bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.010055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930127f777e34dd1911ed0443282d41a, user_id=f489c80ce3324b818096ede43b690fb4, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.012816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eca3e2-ddfc-449e-964e-f63c1bcf9372 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.034352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-17a81ec1-4574-4e2d-b5d5-3ffaba7489bb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.035688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-17a81ec1-4574-4e2d-b5d5-3ffaba7489bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.044223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 807/1607] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.049256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab877a-2b31-41b3-861b-03f28fca73a9 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.049541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab877a-2b31-41b3-861b-03f28fca73a9 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.049850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab877a-2b31-41b3-861b-03f28fca73a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.078296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7fab877a-2b31-41b3-861b-03f28fca73a9 None None] MFA Rules not processed for user `797d321bbca441ebb7ed7d8db3b09c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.087520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7fab877a-2b31-41b3-861b-03f28fca73a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.114681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 801/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:30 2022] PUT /identity/v3/projects/930127f777e34dd1911ed0443282d41a/users/f489c80ce3324b818096ede43b690fb4/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:31.116101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.117888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.119277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.140767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.145725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.167450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 808/1609] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.168953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.169426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.169910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.200968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] MFA Rules not processed for user `d28e4b47b1fb4f24bdbe2ef9c99a6b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.202562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.213435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:31.239114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 802/1610] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.240003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.279909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 809/1611] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.281037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.301344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6f894-99d1-4e9a-a45e-8b639c5608f3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.306187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.306709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:31.307615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.309372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.310155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.310907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.314551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.330675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a65b-13f3-445a-9f9e-6dbe737f47bd None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.342743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.343477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.343859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.344333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.354357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.355886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.366888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 803/1612] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:31.367642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.377153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec8949-23f9-49bc-a3bb-9364c2423f10 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.381917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.386254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.386254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.386254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.392492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.394077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf439b6-cf12-4103-aceb-75a58073b2f5 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.406279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 804/1613] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:31.407715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.407945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.408267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.415728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 810/1614] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:31.415728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.432613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.433998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.512488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5366b82-c565-4293-986f-52d919ab8f71 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.517662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 805/1615] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:31.518989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.519140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.519635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.520030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.520485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.521799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.524206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.598282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31d2e0-3cb5-4e4b-abd7-53cd5f90c723 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.602034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.602662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.602940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.603213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.614605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.619491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d65af-8f8b-4263-8c21-9056f00e266e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.628264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 806/1616] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:31.640896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc71c-8dbf-401c-8b01-339e4333caac None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.642174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc71c-8dbf-401c-8b01-339e4333caac None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.643270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc71c-8dbf-401c-8b01-339e4333caac None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.650298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 807/1617] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 14:35:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:31.650298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dc71c-8dbf-401c-8b01-339e4333caac None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.661577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 811/1618] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/projects => generated 405 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:31.664564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.671337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dc71c-8dbf-401c-8b01-339e4333caac None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.748761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.750089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.750450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.750818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.751795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.754010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.755151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da62ccb-009e-442b-9029-51dfe0908ff4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.765432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.767571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.770166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.770166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.784825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.787017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571ad99-2845-4709-b0ad-09b949541bc1 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.800119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 812/1619] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:31.801931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.802470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.803005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.854135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 808/1620] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:31.858447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:31.884377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10823776-0f77-43c9-8117-7170ba9cf0eb admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:31.889069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:31.898656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:31.899305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:31.899587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:31.899862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:31.900553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:31.906311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:31.910557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:31.912631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:31.921912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:32.017161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 813/1621] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:32.024980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.049040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504a53-e226-49e2-a6b3-fc74aca352e1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.053081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 809/1622] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:31 2022] POST /identity/v3/users => generated 309 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:32.054040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.060305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.062324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.062803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.063616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.065454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.067786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.081692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f412-6603-4084-9656-0dc59eeafba3 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.102359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.103348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.104033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.104546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.110106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 814/1623] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:32.112328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.130663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a39e4c-2bf7-4a81-9d75-ca43da41860c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.145237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.146564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.147472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.148802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.236772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.236772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 815/1624] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:32.236772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.236772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e6b2-14ed-48ee-baa9-70fd113c07e3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.262418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.262418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.262733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.262733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.262733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.262733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] PATH_INFO: `/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba/users/28aa1561b95444c38859f6902b293670/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.263307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 810/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:32.263307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.277980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c918b-c603-47e4-86a1-513b5a7aa8e4 service nova] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.295529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.299320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.299849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.300234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] PATH_INFO: `/v3/projects/c8b47289917840a79fce5d29ea7e25e1/users/a10843fa1e434d95b6459aefd5a3e772/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.301076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15a3313f3594900ae8ddc3b0fd1c5ba, user_id=28aa1561b95444c38859f6902b293670, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.310403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.337051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b47289917840a79fce5d29ea7e25e1, user_id=a10843fa1e434d95b6459aefd5a3e772, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.343773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.434896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 816/1626] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:32 2022] PUT /identity/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba/users/28aa1561b95444c38859f6902b293670/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:32.434896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.468918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 811/1627] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:32 2022] PUT /identity/v3/projects/c8b47289917840a79fce5d29ea7e25e1/users/a10843fa1e434d95b6459aefd5a3e772/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:32.473247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.547664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c844223-e5b4-41e5-8886-742877f43bf2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.644077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebb1dc-ec87-4e7a-b375-f2052f92e21f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.668860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.669803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.669803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.670090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.670774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.672388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.692680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.694677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.694970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.697424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.697837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.701186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.707516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 812/1628] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:32.722487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.744579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 817/1629] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:35:32.745519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:32.758314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299448e-9667-4f6a-a1f7-5f1b73c2306f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.764010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f833e3-cb77-48e3-8d11-ff9e90722873 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:32.769284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.769982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.770262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.770539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] PATH_INFO: `/v3/projects/c8b47289917840a79fce5d29ea7e25e1/users/a10843fa1e434d95b6459aefd5a3e772/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.773025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b47289917840a79fce5d29ea7e25e1, user_id=a10843fa1e434d95b6459aefd5a3e772, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.776913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c464e-4c16-4a4f-8bca-62f84b855030 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.780650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:32.781371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.781706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.786256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] PATH_INFO: `/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba/users/28aa1561b95444c38859f6902b293670/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.786256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15a3313f3594900ae8ddc3b0fd1c5ba, user_id=28aa1561b95444c38859f6902b293670, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:32.788244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c4154-190f-4d9f-a7b7-4e192029ed83 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:32.811583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 813/1630] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:32 2022] PUT /identity/v3/projects/c8b47289917840a79fce5d29ea7e25e1/users/a10843fa1e434d95b6459aefd5a3e772/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:32.816062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.816358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.816637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.835620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 818/1631] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:32 2022] PUT /identity/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba/users/28aa1561b95444c38859f6902b293670/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:32.837984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2da641-d50c-4742-af67-263f5de7794f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:32.837984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2da641-d50c-4742-af67-263f5de7794f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:32.837984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2da641-d50c-4742-af67-263f5de7794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:32.892026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8d2da641-d50c-4742-af67-263f5de7794f None None] MFA Rules not processed for user `28aa1561b95444c38859f6902b293670`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:32.895643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8d2da641-d50c-4742-af67-263f5de7794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:32.900703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] MFA Rules not processed for user `a10843fa1e434d95b6459aefd5a3e772`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:32.901974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:32.955498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:33.023938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:33.097092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 814/1632] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:33.105974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:33.108886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 819/1633] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:33.116318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251affbc-aa0d-4a3b-8f7e-bec6291a31d1 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.116505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251affbc-aa0d-4a3b-8f7e-bec6291a31d1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.116960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251affbc-aa0d-4a3b-8f7e-bec6291a31d1 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.118552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 820/1634] 10.222.0.49 () {54 vars in 962 bytes} [Fri Jul 29 14:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:33.123492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251affbc-aa0d-4a3b-8f7e-bec6291a31d1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:33.199949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437927b3-63c6-4301-860f-885bcf1976fa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:33.211354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:33.213585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.214170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.216122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.238473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:33.243046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289bfce-2bde-4d89-bf54-45d0452818ff service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:33.257746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251affbc-aa0d-4a3b-8f7e-bec6291a31d1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:33.262677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:33.264150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.264530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 815/1635] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:33.265994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8a5e-9a10-4ff1-8d2b-c21c503e2d16 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.266289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8a5e-9a10-4ff1-8d2b-c21c503e2d16 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.266571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8a5e-9a10-4ff1-8d2b-c21c503e2d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.279920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.280403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.303946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:33.308036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb6d2b-b73d-4d44-92ed-73cc8b0c6794 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:33.310722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4ebd8a5e-9a10-4ff1-8d2b-c21c503e2d16 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:33.315202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4ebd8a5e-9a10-4ff1-8d2b-c21c503e2d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:33.334608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 821/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:33.468057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 816/1637] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Jul 29 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:33.473823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707effe-8234-40b4-bcef-d9cb20e49e43 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.474014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707effe-8234-40b4-bcef-d9cb20e49e43 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.474297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707effe-8234-40b4-bcef-d9cb20e49e43 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.475683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 822/1638] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 14:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:33.494874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.494874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.499398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.525269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:33.526592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:33.533689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707effe-8234-40b4-bcef-d9cb20e49e43 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:33.561899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 817/1639] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:33.576708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:33.587650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69dc29-670d-4759-81b7-d0c69c22581f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:33.596461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:33.597152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.597435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.597711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.625504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:33.627683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:33.653395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 818/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:35:33.654745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:33.674720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707effe-8234-40b4-bcef-d9cb20e49e43 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:33.675567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88c36b-e86b-4796-8445-ce6c701a749b service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:33.680547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:33.681225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.681508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.681784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] PATH_INFO: `/v3/users/797d321bbca441ebb7ed7d8db3b09c38` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.683638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=797d321bbca441ebb7ed7d8db3b09c38)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:33.686526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:33.689370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:33.691505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:33.692796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:33.692796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] PATH_INFO: `/v3/users/78e1329dc558419dba83bd62ea7d69c1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:33.696417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e1329dc558419dba83bd62ea7d69c1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:33.698713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: WARNING py.warnings [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:33.698713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: warnings.warn(msg) Jul 29 14:35:33.698713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: Jul 29 14:35:33.701390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:34.195206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797d321bbca441ebb7ed7d8db3b09c38'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.195206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797d321bbca441ebb7ed7d8db3b09c38'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.195206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797d321bbca441ebb7ed7d8db3b09c38'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.195206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797d321bbca441ebb7ed7d8db3b09c38'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.198184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e1329dc558419dba83bd62ea7d69c1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.198184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e1329dc558419dba83bd62ea7d69c1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.260680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9b5bf541-1427-4ee5-9048-94633bdc0e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797d321bbca441ebb7ed7d8db3b09c38'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.271182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 819/1641] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:35:33 2022] DELETE /identity/v3/users/797d321bbca441ebb7ed7d8db3b09c38 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:34.271182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159839-55d2-46fd-966e-98f3868cd7d0 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.271182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159839-55d2-46fd-966e-98f3868cd7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.271182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159839-55d2-46fd-966e-98f3868cd7d0 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.271837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 820/1642] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:34.273985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd4a9-4f08-46b6-a648-5e73c69de9a3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.273985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd4a9-4f08-46b6-a648-5e73c69de9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.274453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd4a9-4f08-46b6-a648-5e73c69de9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.295110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8dddd4a9-4f08-46b6-a648-5e73c69de9a3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:34.296166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8dddd4a9-4f08-46b6-a648-5e73c69de9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.346374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e1329dc558419dba83bd62ea7d69c1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.369389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e1329dc558419dba83bd62ea7d69c1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.371258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fa830685-6fdc-4635-aec0-a90bbd2a7548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e1329dc558419dba83bd62ea7d69c1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:34.385124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 821/1643] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:34.386902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.387347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.387775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.395973 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 823/1644] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:35:33 2022] DELETE /identity/v3/users/78e1329dc558419dba83bd62ea7d69c1 => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:34.401411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.401713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.401997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.412193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:34.414130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.440454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:34.442328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.530792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 822/1645] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:34.532116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:34.534860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.564973 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 824/1646] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:34.571819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:34.571819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.635025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb9995-1127-47db-ad48-ca7f256a2e80 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:34.636410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727974d2-7fac-424d-be95-b8bf918b0077 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:34.643743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:34.644391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.644676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.644957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.646125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:34.646814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.647097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.647493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.671928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:34.675529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:34.677688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:34.680438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:34.708460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 825/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:34.709417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:34.711638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 823/1648] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:34.721672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:34.730250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282724e-d0f8-4e08-a7e8-66319c054599 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:34.741697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:34.744295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.745247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.745780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.761883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:34.763731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a529a4c-894d-4d36-8f51-2b1ef92ca7a6 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:34.778719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 826/1649] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:34.844119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:34.935846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:34.937300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:34.951669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f6cc-4329-40c3-8b8e-a9e84e7651b4 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:34.958148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:34.958752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:34.959032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:34.959308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] PATH_INFO: `/v3/users/d28e4b47b1fb4f24bdbe2ef9c99a6b95` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:34.968488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28e4b47b1fb4f24bdbe2ef9c99a6b95)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:34.972165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:35.221298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28e4b47b1fb4f24bdbe2ef9c99a6b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:35.235487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28e4b47b1fb4f24bdbe2ef9c99a6b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:35.245168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28e4b47b1fb4f24bdbe2ef9c99a6b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:35.250206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28e4b47b1fb4f24bdbe2ef9c99a6b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:35.355539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28e4b47b1fb4f24bdbe2ef9c99a6b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:35.372198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 824/1650] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:35:34 2022] DELETE /identity/v3/users/d28e4b47b1fb4f24bdbe2ef9c99a6b95 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:35.376680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:35.377229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:35.377229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:35.409810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:35.411052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:35.507025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 827/1651] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:35.522214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:35.524136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:35.608201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d9bfa-e1a2-466d-9677-fcee19f7b4fc admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:35.615624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:35.617358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:35.617728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:35.618477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:35.630721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:35.632363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:35.659929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 825/1652] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:36.211505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:36.243420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:37.052530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:37.059093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:37.100857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1e17-b501-4b96-8ceb-c6b80e5da950 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:37.123419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:37.130131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:37.135118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:37.136754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:37.137033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:37.137427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] PATH_INFO: `/v3/projects/3ee32ab01d4749ff8f2ae792045c2bcb` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:37.143817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee32ab01d4749ff8f2ae792045c2bcb)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:37.148349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:37.166834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6239e-c5b6-4425-bece-7967c8828ac1 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:37.181890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:37.182909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:37.183193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:37.183525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] PATH_INFO: `/v3/projects/c8e78f4f35e54ff9bc37e22705074bcf` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:37.185795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e78f4f35e54ff9bc37e22705074bcf)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:37.186835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: WARNING py.warnings [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 29 14:35:37.186835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: warnings.warn(msg) Jul 29 14:35:37.186835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: Jul 29 14:35:37.192673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:37.364820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee32ab01d4749ff8f2ae792045c2bcb', 'request_id': 'req-3b7296a3-742a-47a7-ad95-08efa3d4cab4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.442952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e78f4f35e54ff9bc37e22705074bcf', 'request_id': 'req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.477269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee32ab01d4749ff8f2ae792045c2bcb', 'request_id': 'req-3b7296a3-742a-47a7-ad95-08efa3d4cab4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.492881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee32ab01d4749ff8f2ae792045c2bcb', 'request_id': 'req-3b7296a3-742a-47a7-ad95-08efa3d4cab4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.496676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e78f4f35e54ff9bc37e22705074bcf', 'request_id': 'req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.524152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] The token cache is being invalidate because project 3ee32ab01d4749ff8f2ae792045c2bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:35:37.524650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.528070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.534570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 828/1653] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:35:37 2022] DELETE /identity/v3/projects/3ee32ab01d4749ff8f2ae792045c2bcb => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:37.536374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:37.538641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:37.592692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e78f4f35e54ff9bc37e22705074bcf', 'request_id': 'req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.614496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] The token cache is being invalidate because project c8e78f4f35e54ff9bc37e22705074bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:35:37.615312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.616493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:37.628198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 826/1654] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:35:37 2022] DELETE /identity/v3/projects/c8e78f4f35e54ff9bc37e22705074bcf => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:35:37.629801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:37.633088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:37.740336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:37.754038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7296a3-742a-47a7-ad95-08efa3d4cab4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:37.818664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d823818-f96a-46f8-bcb9-d5c5eb83ec26 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:37.822637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:37.827451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:37.828737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:37.831863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] PATH_INFO: `/v3/users/a10843fa1e434d95b6459aefd5a3e772` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:37.832777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:37.833764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:37.834693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:37.834881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10843fa1e434d95b6459aefd5a3e772)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:37.835275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] PATH_INFO: `/v3/users/28aa1561b95444c38859f6902b293670` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:37.836747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28aa1561b95444c38859f6902b293670)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:37.839030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:37.847673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:38.116342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28aa1561b95444c38859f6902b293670'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.135697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28aa1561b95444c38859f6902b293670'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.155237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:38.159053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28aa1561b95444c38859f6902b293670'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.160359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28aa1561b95444c38859f6902b293670'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.160701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10843fa1e434d95b6459aefd5a3e772'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.199894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10843fa1e434d95b6459aefd5a3e772'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.257033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28aa1561b95444c38859f6902b293670'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.279360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 827/1655] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:35:37 2022] DELETE /identity/v3/users/28aa1561b95444c38859f6902b293670 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:38.280616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:38.286386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:38.383561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10843fa1e434d95b6459aefd5a3e772'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.396022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10843fa1e434d95b6459aefd5a3e772'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.397235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-651b9d4e-51b3-497f-b3d0-21ff75d96e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10843fa1e434d95b6459aefd5a3e772'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.424432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 829/1656] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:35:37 2022] DELETE /identity/v3/users/a10843fa1e434d95b6459aefd5a3e772 => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:38.520445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac5ac-2e87-406d-ac3c-1af20c4e2f00 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:38.714646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:38.715328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:38.715607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:38.715879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] PATH_INFO: `/v3/projects/7d4c357515b04f71ba99c3f1cae80ddd` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:38.718435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4c357515b04f71ba99c3f1cae80ddd)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:38.723101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:38.960447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4c357515b04f71ba99c3f1cae80ddd', 'request_id': 'req-7867ad1b-091f-4db7-9811-8a0dcde83759'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:38.976312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4c357515b04f71ba99c3f1cae80ddd', 'request_id': 'req-7867ad1b-091f-4db7-9811-8a0dcde83759'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:39.084080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4c357515b04f71ba99c3f1cae80ddd', 'request_id': 'req-7867ad1b-091f-4db7-9811-8a0dcde83759'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:39.099586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] The token cache is being invalidate because project 7d4c357515b04f71ba99c3f1cae80ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:35:39.100171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:39.102606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7867ad1b-091f-4db7-9811-8a0dcde83759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:39.108676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 828/1657] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:35:38 2022] DELETE /identity/v3/projects/7d4c357515b04f71ba99c3f1cae80ddd => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:39.114858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:39.115260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:39.115416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:39.149121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:39.156060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:39.256780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 830/1658] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:39.264805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:39.265546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:39.266887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:39.299090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:39.301880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:39.322017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 829/1659] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:39.326040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:39.340497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc162b5-6f06-429d-9acb-5a0bcdb8c754 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:39.352036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:39.352971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:39.353435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:39.354085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:39.355119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:39.355740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:39.359049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:39.376340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 831/1660] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:39.380807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:39.391609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5e135-6b7f-472f-acb2-8f02f29d121c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:39.401322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:39.408905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:39.410139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:39.420326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:39.445905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:39.445905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:39.448781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:39.621563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 830/1661] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:39 2022] POST /identity/v3/projects => generated 357 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:35:39.625632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:39.641094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600994d3-4c85-4fe8-882b-32e192f2d1a7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:39.779845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:39.780656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:39.783117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:39.786677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:39.787977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:39.792386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:39.962677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 832/1662] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:39 2022] POST /identity/v3/users => generated 279 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:39.966292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:39.980419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e214b5f-ede8-4db9-b966-e5526411ecf6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:40.000945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:40.000945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.000945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.000945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.001942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:40.003211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:40.038112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 831/1663] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:40.038275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:40.052137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954fd0e-b7c7-496c-9e0a-cbbc1fe957b0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:40.069082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:40.074846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.075157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.075440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.079104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:40.094597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b180e-a26a-4968-8936-a1b0aaec9ee9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:40.136128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:40.136562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 833/1664] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:40.151621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efb4fd-38e5-4762-8473-be934fc46ecb admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:40.174466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:40.186463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.187311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.187789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] PATH_INFO: `/v3/projects/fe967d331dee4898953eae76d7911f5e/users/bdc8ea78e5c049ec8468e9326fa75783/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.219420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe967d331dee4898953eae76d7911f5e, user_id=bdc8ea78e5c049ec8468e9326fa75783, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:40.222627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:40.323958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 832/1665] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:40 2022] PUT /identity/v3/projects/fe967d331dee4898953eae76d7911f5e/users/bdc8ea78e5c049ec8468e9326fa75783/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:40.336715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.337254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.337326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.393318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] MFA Rules not processed for user `bdc8ea78e5c049ec8468e9326fa75783`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:40.396319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:40.425210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:40.512129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 834/1666] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:40 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:40.513230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:40.618676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:40.661857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd166-8fd9-437e-b790-803761552713 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:40.672839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:40.673876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.674413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.675134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.675999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:40.679265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:40.808384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 833/1667] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:40 2022] POST /identity/v3/projects => generated 357 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:40.812325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:40.827849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf36340-843e-4f93-a1bf-091582ee3b54 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:40.969846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:40.970547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:40.970830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:40.971180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:40.971890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:40.977480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.092590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 835/1668] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:40 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:35:41.098958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.111936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d7128-8799-4034-8228-8f93a3d50bff admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:41.129875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:41.130533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.130819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.132429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.132429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:41.133579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2cad-f72a-49c9-b3eb-bf13dd0b75b7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.155730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 834/1669] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:41.163175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.175872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6831-c09b-416e-808b-25c6a42f12d6 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:41.186161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.186557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.186843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.194955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:41.196173 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.196720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.197907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] PATH_INFO: `/v3/projects/845fb99b4ce548b0bce4fe5da7188702/users/e8035e5fca0540cc8e15a8e76a4fe039/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.222732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=845fb99b4ce548b0bce4fe5da7188702, user_id=e8035e5fca0540cc8e15a8e76a4fe039, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:41.227045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.248471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] MFA Rules not processed for user `de769d721b8b49c6a5d7d391412da122`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:41.249794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:41.324507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 836/1670] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:41 2022] PUT /identity/v3/projects/845fb99b4ce548b0bce4fe5da7188702/users/e8035e5fca0540cc8e15a8e76a4fe039/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:41.325904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.402847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 835/1671] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:41.405921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.408179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:41.431730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f619465-d0e1-4147-8102-af74b77f1776 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:41.493869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7bdba-411c-448d-bffa-1f6b70b75197 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:41.506227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:41.506878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.507372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.507808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.508599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:41.514406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.540573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 837/1672] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:41.541401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.549643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:41.565762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:41.566525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.567251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.567595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] PATH_INFO: `/v3/projects/c8b47289917840a79fce5d29ea7e25e1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.570825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b47289917840a79fce5d29ea7e25e1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:41.575166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.597523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:41.729706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9074b-6a69-4440-9d69-10f9eb666aa3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:41.745950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:41.746594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:41.746943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:41.747187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:41.774157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b47289917840a79fce5d29ea7e25e1', 'request_id': 'req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:41.795056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b47289917840a79fce5d29ea7e25e1', 'request_id': 'req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:41.839284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:41.881945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:41.883264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:41.895022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b47289917840a79fce5d29ea7e25e1', 'request_id': 'req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:41.912175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 838/1673] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:41.914534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] The token cache is being invalidate because project c8b47289917840a79fce5d29ea7e25e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:35:41.914737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:41.917401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.917518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:41.927086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:41.927852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 836/1674] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:35:41 2022] DELETE /identity/v3/projects/c8b47289917840a79fce5d29ea7e25e1 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:41.928884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:41.931780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:41.970166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18bd8-92ca-433b-b18d-5a87643ad89e service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.146688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.146907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20cd9-c706-4c4f-aec8-9a67e79abdc8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.147414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.147704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.147982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] PATH_INFO: `/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.151920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15a3313f3594900ae8ddc3b0fd1c5ba)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.164896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.168274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.168274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.168274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] PATH_INFO: `/v3/projects/845fb99b4ce548b0bce4fe5da7188702/users/e8035e5fca0540cc8e15a8e76a4fe039/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.169283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=845fb99b4ce548b0bce4fe5da7188702, user_id=e8035e5fca0540cc8e15a8e76a4fe039, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.169851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.172068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8277038-5c2c-4582-bd85-31364f530e55 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.210507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 837/1675] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:41 2022] PUT /identity/v3/projects/845fb99b4ce548b0bce4fe5da7188702/users/e8035e5fca0540cc8e15a8e76a4fe039/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:42.210507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2995e2b-af65-438a-afbe-9e2bfa97c320 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.210507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2995e2b-af65-438a-afbe-9e2bfa97c320 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.210507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2995e2b-af65-438a-afbe-9e2bfa97c320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.250193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a2995e2b-af65-438a-afbe-9e2bfa97c320 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:42.256428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a2995e2b-af65-438a-afbe-9e2bfa97c320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.335146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15a3313f3594900ae8ddc3b0fd1c5ba', 'request_id': 'req-de3667f8-a32a-4af1-9307-25e4c8e13995'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:42.397524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 838/1676] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:42.399843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14aaec-d285-409f-84cd-6d703227c076 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.400222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14aaec-d285-409f-84cd-6d703227c076 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.400582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14aaec-d285-409f-84cd-6d703227c076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.406661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15a3313f3594900ae8ddc3b0fd1c5ba', 'request_id': 'req-de3667f8-a32a-4af1-9307-25e4c8e13995'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:42.408806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:42.417622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15a3313f3594900ae8ddc3b0fd1c5ba', 'request_id': 'req-de3667f8-a32a-4af1-9307-25e4c8e13995'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:42.448318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] The token cache is being invalidate because project c15a3313f3594900ae8ddc3b0fd1c5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:35:42.448620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:42.452102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de3667f8-a32a-4af1-9307-25e4c8e13995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:35:42.457062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 839/1677] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:35:41 2022] DELETE /identity/v3/projects/c15a3313f3594900ae8ddc3b0fd1c5ba => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:42.461818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.462178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.462460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.465882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3f14aaec-d285-409f-84cd-6d703227c076 None None] MFA Rules not processed for user `e8035e5fca0540cc8e15a8e76a4fe039`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:42.467160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3f14aaec-d285-409f-84cd-6d703227c076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.494612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:42.496000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.584277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 839/1678] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:42.584277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.584277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.584277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.597882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 840/1679] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:42.601205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:42.602515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.617025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:42.619500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.643347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 840/1680] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:42.644289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:42.667811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef12a1-a91d-44aa-9e32-9c1a444efff9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.681567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.682473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.683588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.685976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.686994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:42.687313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.690011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacf5bb-9434-40f8-a8eb-e32f30e126f5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.725049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 841/1681] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:42.726419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.726916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.727202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.753414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbb339-fa22-4f7a-9adf-9afd4b1114b2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.766964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.767915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.768239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.768553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.778726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:42.781043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:42.788939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.793374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.805543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 842/1682] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:42.806244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:42.831584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 841/1683] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:42 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:42.834475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:42.847740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765a447-09d4-40b2-b4af-48cf15901f33 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.861156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.863866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.865785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29b4f-9370-43b9-b488-5b3aaa9be6f5 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.865981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.866267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.868669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.883097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.888616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.890203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.890499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.890779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.891560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:35:42.891892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.895853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:42.919282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 842/1684] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:35:42.931519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:42.953299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf21b03-ef1c-47b3-a705-9c95ea11e1b3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:42.963063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:42.963711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:42.964129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:42.964924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:42.965965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:42.968203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.012392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:43.051705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 843/1685] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/projects => generated 405 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:43.056090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.110093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 843/1686] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:42 2022] POST /identity/v3/projects => generated 409 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:43.116645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.194154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125a78-e193-4df1-a083-efcd7b4c3d2d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.315229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba637f-084b-407b-b283-2862f4c881d8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.326316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.332212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.332502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.332931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.333750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.335730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.375909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.377246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.377542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.377871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.379037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.381106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.459928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 844/1687] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:43 2022] POST /identity/v3/users => generated 305 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:43.460625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.488906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 844/1688] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:43 2022] POST /identity/v3/users => generated 303 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:43.493018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.514156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f54b-598d-4ca2-9980-08f9c1f9dea8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.535237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.538696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.540095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.540460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.541556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.543416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.559964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 845/1689] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:43.559964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.570721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15c0f-0e5b-4f46-a7be-c764f494a3a8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.575754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.576458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.577974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.578528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.579920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.581812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.597586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 846/1690] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:43.610398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.621479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3211169-49e9-4906-8004-d66843a8769a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.626383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.627103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.627452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.627818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.628589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.630510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.631772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e310-786b-4cf0-8adb-cc765fd49ca7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.647108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 847/1691] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:43.647108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.650844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.651725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.652179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.652627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.654914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:43.658868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f9e42-1e0f-4a9f-be46-67c688f1de02 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.667240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.668105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.668456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.668806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.669682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.671573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.689703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 848/1692] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:43.690944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.719595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac06eb0-2b6a-4a6f-b78f-92919e947aa5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.725416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.726106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.726397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.726679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] PATH_INFO: `/v3/projects/6eed875a70e9411785d28bdbf7e8bc55/users/1ea6ddc49e7f411dad9408789c4cb8b6/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.758492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eed875a70e9411785d28bdbf7e8bc55, user_id=1ea6ddc49e7f411dad9408789c4cb8b6, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.760894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23ec11-4ab1-488a-8bee-285e09d17c5c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.787711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.789458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.805683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 845/1693] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:43 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:43.807623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:43.828238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c436d-e82b-48a9-8528-4209e528847c service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:43.836875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:43.838024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.842344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.842344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] PATH_INFO: `/v3/projects/738f33ea3d624445989cbc918a736075/users/6843682103ab4ecdbb03d3ae23a3042b/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.872664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 849/1694] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:43 2022] PUT /identity/v3/projects/6eed875a70e9411785d28bdbf7e8bc55/users/1ea6ddc49e7f411dad9408789c4cb8b6/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:43.873564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738f33ea3d624445989cbc918a736075, user_id=6843682103ab4ecdbb03d3ae23a3042b, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:43.878646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32129f-5080-4f33-bf38-25113efeaf37 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:43.883949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:43.884297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:43.884570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:43.931281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] MFA Rules not processed for user `1ea6ddc49e7f411dad9408789c4cb8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:43.934182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:44.002149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 846/1695] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:43 2022] PUT /identity/v3/projects/738f33ea3d624445989cbc918a736075/users/6843682103ab4ecdbb03d3ae23a3042b/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:44.004757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.005140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.005417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:44.059108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:44.060353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:44.113992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 850/1696] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:44.120569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:44.252766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 847/1697] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:44.303718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:44.326441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fd8dd-df41-4123-ab7c-d527556402f6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:44.338657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:44.341023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.341339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.342855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:44.346387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:44.350687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:44.392847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:44.394385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:44.508564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a94d9-b77b-46c9-85a0-59ecd4a83ab9 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:44.521409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 851/1698] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:44 2022] POST /identity/v3/projects => generated 409 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:44.522203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:44.522870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.523158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.525245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:44.526482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:44.569816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:44.591909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:44.643961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 848/1699] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:44.669587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab5c69-074e-40ab-aae7-c951228e17a0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:44.680251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:44.680952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.681260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.681535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:44.682578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:44.685785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:44.831388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 852/1700] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:44 2022] POST /identity/v3/users => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:44.836966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:44.859216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d394e46-017f-468e-a712-4f990da1d0d2 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:44.867641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:44.868278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.868692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.868926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:44.879686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:44.881392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:44.922762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 849/1701] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:44.927452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:44.946988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb976a7-f3d4-42f2-8732-bfc8282af482 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:44.962260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:44.963039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:44.963319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:44.963592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] PATH_INFO: `/v3/projects/2e6c6537708a42fda922893df4f61036/users/e6a82817a0f048eab267eda75cd99c1c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.001133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6c6537708a42fda922893df4f61036, user_id=e6a82817a0f048eab267eda75cd99c1c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:45.014010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:45.021140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:45.114473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 853/1702] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:44 2022] PUT /identity/v3/projects/2e6c6537708a42fda922893df4f61036/users/e6a82817a0f048eab267eda75cd99c1c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:45.118761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:45.136278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db965ed3-f08c-47d1-890f-7efd2595ab02 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:45.245225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:45.245874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:45.247321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:45.247871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.249811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:45.250474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30144d48-c9a4-472e-8900-d9ac142ebd7a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:45.266317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 850/1703] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:45.271069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:45.284485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eccecf-88cd-481b-9a7d-0cb366f41aaf admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:45.289491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:45.290177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:45.290458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:45.290734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] PATH_INFO: `/v3/projects/2e6c6537708a42fda922893df4f61036/users/e6a82817a0f048eab267eda75cd99c1c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.293236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6c6537708a42fda922893df4f61036, user_id=e6a82817a0f048eab267eda75cd99c1c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:45.295968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:45.320084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 854/1704] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:45 2022] PUT /identity/v3/projects/2e6c6537708a42fda922893df4f61036/users/e6a82817a0f048eab267eda75cd99c1c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:45.321594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a8a5e-9f8e-482b-8e5c-08027580da41 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:45.322009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a8a5e-9f8e-482b-8e5c-08027580da41 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:45.322343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a8a5e-9f8e-482b-8e5c-08027580da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.358081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fe0a8a5e-9f8e-482b-8e5c-08027580da41 None None] MFA Rules not processed for user `e6a82817a0f048eab267eda75cd99c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:45.359678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fe0a8a5e-9f8e-482b-8e5c-08027580da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:45.520727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 851/1705] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:45.531763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:45.624688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb393c-5603-4f2a-b2ff-4b8080fdbd1c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:45.631064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:45.631561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:45.631841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:45.632111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.657713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:45.658230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:45.658447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:45.673021 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:45.678678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:45.707870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 855/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:45.772845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] MFA Rules not processed for user `30432a068ad8486db8af635e66776f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:45.780417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:45.834161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:45.996529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 852/1707] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:46.013422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:46.015675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:46.183733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eb9f7-a1e5-4030-89a7-ee0d4c9d818f service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:46.198567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:46.200388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:46.200388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:46.200388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:46.232829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:46.232829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551f3d-878e-4a44-8afa-8350fdf3cb6a service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:46.253932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 856/1708] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:46.255423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:46.480160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:46.498897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147bff0-e5f6-4e65-9084-194dc83a7ba9 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:46.523892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:46.523892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:46.523892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:46.523892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:46.681683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:46.688176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3591631-d6c1-4c48-af1a-a54cce39ae89 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:46.711085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 853/1709] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:47.132704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:48.195623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:48.196217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:48.196217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:48.258720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] MFA Rules not processed for user `f4ae3fb6a7954a24b88df4f35f55020c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:48.264463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:48.305386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:48.445289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 857/1710] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:48.456699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3415f9-825b-403d-b707-ac4ef11cd2f2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:48.457232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3415f9-825b-403d-b707-ac4ef11cd2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:48.457414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3415f9-825b-403d-b707-ac4ef11cd2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:48.514274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bb3415f9-825b-403d-b707-ac4ef11cd2f2 None None] MFA Rules not processed for user `f4ae3fb6a7954a24b88df4f35f55020c`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:48.515599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bb3415f9-825b-403d-b707-ac4ef11cd2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:48.546117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 854/1711] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:48.558446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:48.559942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:48.587062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322b86-108b-4d62-a600-e2c29ddc12be None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:48.595983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:48.596821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:48.597396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:48.626079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:48.650832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:48.654411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebd191-e5f2-451e-8f81-54c890ec861c service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:48.678726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 858/1712] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:35:50.943107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe455b-20c0-49fa-9d51-2a3965d1a7d3 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:50.943107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe455b-20c0-49fa-9d51-2a3965d1a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:50.943107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe455b-20c0-49fa-9d51-2a3965d1a7d3 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:50.943107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 855/1713] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 14:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e668-58b3-4699-8f01-e1fbe1fad600 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e668-58b3-4699-8f01-e1fbe1fad600 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e668-58b3-4699-8f01-e1fbe1fad600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-15d4e668-58b3-4699-8f01-e1fbe1fad600 None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-15d4e668-58b3-4699-8f01-e1fbe1fad600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:50.951648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 859/1714] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:50.966022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe455b-20c0-49fa-9d51-2a3965d1a7d3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:50.966022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe455b-20c0-49fa-9d51-2a3965d1a7d3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:50.966022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:50.966022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:50.967342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:50.967342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:50.967342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:50.967342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4fafb-f95b-4c17-ad5d-a45c986cbcc7 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:50.967342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 856/1715] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:51.200533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86663e-828a-4ce5-a3d0-ae9b4c1ae5c5 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:51.200968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86663e-828a-4ce5-a3d0-ae9b4c1ae5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:51.201345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86663e-828a-4ce5-a3d0-ae9b4c1ae5c5 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:51.203114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 860/1716] 10.222.0.49 () {54 vars in 961 bytes} [Fri Jul 29 14:35:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:51.209088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368942b-f733-48de-b874-dfea2a39387a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:51.209301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368942b-f733-48de-b874-dfea2a39387a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:51.209949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368942b-f733-48de-b874-dfea2a39387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:51.240756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b368942b-f733-48de-b874-dfea2a39387a None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:51.242884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b368942b-f733-48de-b874-dfea2a39387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:51.392158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a15503-dd19-4170-a4c7-cfb5375abd49 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:51.392366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a15503-dd19-4170-a4c7-cfb5375abd49 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:51.392809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a15503-dd19-4170-a4c7-cfb5375abd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:51.398635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 857/1717] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Jul 29 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:51.406533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368942b-f733-48de-b874-dfea2a39387a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:51.445459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368942b-f733-48de-b874-dfea2a39387a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:51.450120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-54a15503-dd19-4170-a4c7-cfb5375abd49 None None] MFA Rules not processed for user `f4ae3fb6a7954a24b88df4f35f55020c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:51.456278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-54a15503-dd19-4170-a4c7-cfb5375abd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:51.480838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:51.486945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:51.486945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:51.486945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:51.491259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:51.497197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 861/1718] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:51.537245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:51.539837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:51.567351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95988987-f912-4210-9759-0e7e68c0fbef None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:51.569451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95988987-f912-4210-9759-0e7e68c0fbef None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:51.569729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95988987-f912-4210-9759-0e7e68c0fbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:51.574892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 858/1719] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:51.658091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-95988987-f912-4210-9759-0e7e68c0fbef None None] MFA Rules not processed for user `6843682103ab4ecdbb03d3ae23a3042b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:51.659988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-95988987-f912-4210-9759-0e7e68c0fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:51.900250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 862/1720] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:51 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:51.901577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:51.996217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:52.159364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6eca-79e3-48b3-857f-ac45f23b031d service placement] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:52.166323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd30c9-128f-4ffb-a074-2754423bbe1c None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.166323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd30c9-128f-4ffb-a074-2754423bbe1c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.166323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd30c9-128f-4ffb-a074-2754423bbe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.171715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:52.173456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.173762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.174097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.174856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:52.176919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:52.229594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-16cd30c9-128f-4ffb-a074-2754423bbe1c None None] MFA Rules not processed for user `de769d721b8b49c6a5d7d391412da122`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:52.231334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-16cd30c9-128f-4ffb-a074-2754423bbe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:52.303918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:52.307835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 859/1721] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:35:51 2022] POST /identity/v3/projects => generated 405 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:52.311923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:52.391127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 863/1722] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:52.485487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3af9-98b7-40ec-996d-c75f0a3b4b7d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:52.499776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:52.500721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.501191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.501620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.502452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:52.505200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:52.590495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.591087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.591516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.633101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 860/1723] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:35:52 2022] POST /identity/v3/users => generated 309 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:35:52.637473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:52.677889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ab2f-75b1-4f17-b486-3d1a523650d3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:52.698203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:52.701154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.701154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.701154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.701154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:52.703216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:52.717850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 861/1724] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:35:52.719005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] MFA Rules not processed for user `f3a38dc21bc34164aea48065da64823f`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:52.720742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:52.722311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:52.737586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b992-7785-48a9-925d-16de20fe8873 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:52.745440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:52.746325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:52.746745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:52.747172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] PATH_INFO: `/v3/projects/9d5fdc640a7b4fd692408ea43be96dda/users/bff0a86afaed44378a73358db5fd0b95/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:52.770002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fdc640a7b4fd692408ea43be96dda, user_id=bff0a86afaed44378a73358db5fd0b95, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:52.774770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:52.872636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 864/1725] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:52.889406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 862/1726] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:52 2022] PUT /identity/v3/projects/9d5fdc640a7b4fd692408ea43be96dda/users/bff0a86afaed44378a73358db5fd0b95/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:52.893643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:52.928423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95b1f3-f03f-4a62-ad95-a1d31f7c1c79 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:53.272375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:53.274316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:53.275159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:53.275637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:53.276633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:53.279314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c5ac-ebd7-47c1-841c-b95bf657d9e7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:53.298756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 865/1727] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:35:53.304934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:53.323730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb1025-34bf-4d1e-9c05-afba21863857 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:53.334982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:53.335692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:53.335974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:53.336275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] PATH_INFO: `/v3/projects/9d5fdc640a7b4fd692408ea43be96dda/users/bff0a86afaed44378a73358db5fd0b95/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:53.354104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fdc640a7b4fd692408ea43be96dda, user_id=bff0a86afaed44378a73358db5fd0b95, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:53.368784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d06200-5bd1-42be-8982-aee34afba1fb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:53.411475 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 863/1728] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:35:53 2022] PUT /identity/v3/projects/9d5fdc640a7b4fd692408ea43be96dda/users/bff0a86afaed44378a73358db5fd0b95/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:35:53.506883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:53.507170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:53.507445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:53.509507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 866/1729] 10.222.0.49 () {54 vars in 912 bytes} [Fri Jul 29 14:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:53.518374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce9fe9-43b4-4c83-8837-a4eb9a4566c4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:53.518716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce9fe9-43b4-4c83-8837-a4eb9a4566c4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:53.519000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce9fe9-43b4-4c83-8837-a4eb9a4566c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:53.565990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c6ce9fe9-43b4-4c83-8837-a4eb9a4566c4 None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:53.568286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c6ce9fe9-43b4-4c83-8837-a4eb9a4566c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:53.711325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:53.769225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 864/1730] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:53.895805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:53.897981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:53.939716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46dcef-c07c-4d93-82b1-52073aaf4905 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:53.969532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:53.970351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:53.970692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:53.971049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:54.003496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:54.005207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:54.033467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 867/1731] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:54.725755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:54.726248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:54.726351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:54.740464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:54.811502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:54.814469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] MFA Rules not processed for user `f489c80ce3324b818096ede43b690fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:54.817590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:54.838149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa7cf3-c322-4774-bd43-3a0b9fc0cd70 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:54.845025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:54.845898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:54.846360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:54.846836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:54.947194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:54.949062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83632ad-201f-4cee-8b88-caaf208448ae service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:54.983541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 868/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:55.059644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 865/1733] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:55.074669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e3a3-eaa7-4e7b-8452-a5c3275443f4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:55.074669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e3a3-eaa7-4e7b-8452-a5c3275443f4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:55.074998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e3a3-eaa7-4e7b-8452-a5c3275443f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:55.136517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e053e3a3-eaa7-4e7b-8452-a5c3275443f4 None None] MFA Rules not processed for user `f489c80ce3324b818096ede43b690fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:55.137823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e053e3a3-eaa7-4e7b-8452-a5c3275443f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:55.190673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 869/1734] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:55.339045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:55.356622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa20a0-5fcf-4d1a-b23f-9eddd472a58a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:55.367192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:55.367913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:55.368168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:55.368680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:55.378261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:55.380589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd04a8a-ef2d-4a6a-92a8-267ffe51349e service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:55.412459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 866/1735] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:35:56.508562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9799e-e0f1-4676-bad1-c2537c4b8df4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:56.510654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9799e-e0f1-4676-bad1-c2537c4b8df4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:56.511140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9799e-e0f1-4676-bad1-c2537c4b8df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:56.575078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fe40-2efc-40e8-a2b4-f8602511937e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:56.576771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fe40-2efc-40e8-a2b4-f8602511937e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:56.576771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fe40-2efc-40e8-a2b4-f8602511937e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:56.580731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-74b9799e-e0f1-4676-bad1-c2537c4b8df4 None None] MFA Rules not processed for user `30432a068ad8486db8af635e66776f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:56.587876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-74b9799e-e0f1-4676-bad1-c2537c4b8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:56.667630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9222fe40-2efc-40e8-a2b4-f8602511937e None None] MFA Rules not processed for user `bff0a86afaed44378a73358db5fd0b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:56.669398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9222fe40-2efc-40e8-a2b4-f8602511937e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:56.773707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 870/1736] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:35:56.845702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 867/1737] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:58.698164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdea71-3596-4f2a-9813-013c01298f9d None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:58.698785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdea71-3596-4f2a-9813-013c01298f9d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:58.698785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdea71-3596-4f2a-9813-013c01298f9d None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:58.700091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 871/1738] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:35:58.725894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2e91-3e0d-45b3-9db7-a08e3e7c5455 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:58.726524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2e91-3e0d-45b3-9db7-a08e3e7c5455 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:58.726524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd2e91-3e0d-45b3-9db7-a08e3e7c5455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:58.771037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a3fd2e91-3e0d-45b3-9db7-a08e3e7c5455 None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:35:58.772664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a3fd2e91-3e0d-45b3-9db7-a08e3e7c5455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:35:58.852740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:35:58.869832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 868/1739] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:35:58.880977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdea71-3596-4f2a-9813-013c01298f9d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:35:58.894242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdea71-3596-4f2a-9813-013c01298f9d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:35:58.899768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:35:58.900417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:58.900703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:58.900978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:35:58.927582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:35:58.929347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:35:58.959612 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 872/1740] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:35:59.963022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:35:59.963022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:35:59.963022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:00.050248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] MFA Rules not processed for user `f3a38dc21bc34164aea48065da64823f`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:00.057764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:00.063489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 869/1741] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:00.637324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:00.775968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd008f2-24cd-491a-96dc-b8b100535bf6 service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:00.775968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:00.775968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:00.775968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:00.776600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] PATH_INFO: `/v3/users/6843682103ab4ecdbb03d3ae23a3042b` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:00.776600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6843682103ab4ecdbb03d3ae23a3042b)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:00.776600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:00.776600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:01.015748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6843682103ab4ecdbb03d3ae23a3042b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.031586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6843682103ab4ecdbb03d3ae23a3042b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.050394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6843682103ab4ecdbb03d3ae23a3042b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.050835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6843682103ab4ecdbb03d3ae23a3042b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.144605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6843682103ab4ecdbb03d3ae23a3042b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.156985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 873/1742] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:00 2022] DELETE /identity/v3/users/6843682103ab4ecdbb03d3ae23a3042b => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:01.533968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:01.546041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ffc83-e136-4aa9-9fa2-62c7a52cbc51 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:01.652566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:01.655485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:01.655921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:01.656432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] PATH_INFO: `/v3/projects/738f33ea3d624445989cbc918a736075` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:01.659004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=738f33ea3d624445989cbc918a736075)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:01.661560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:01.836045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738f33ea3d624445989cbc918a736075', 'request_id': 'req-7617de02-776b-4de9-b630-8833e2dbd4cf'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.963035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738f33ea3d624445989cbc918a736075', 'request_id': 'req-7617de02-776b-4de9-b630-8833e2dbd4cf'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:01.970885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738f33ea3d624445989cbc918a736075', 'request_id': 'req-7617de02-776b-4de9-b630-8833e2dbd4cf'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.017754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] The token cache is being invalidate because project 738f33ea3d624445989cbc918a736075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:02.018053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.021372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7617de02-776b-4de9-b630-8833e2dbd4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.025915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 870/1743] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:01 2022] DELETE /identity/v3/projects/738f33ea3d624445989cbc918a736075 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:02.030407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:02.031969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:02.056631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae87c8-8536-4e9e-ae26-8c173cb67129 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:02.205849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:02.207087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:02.208037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:02.208845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] PATH_INFO: `/v3/users/bff0a86afaed44378a73358db5fd0b95` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:02.210929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff0a86afaed44378a73358db5fd0b95)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:02.213384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:02.466223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff0a86afaed44378a73358db5fd0b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.500562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff0a86afaed44378a73358db5fd0b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.517649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff0a86afaed44378a73358db5fd0b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.518414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff0a86afaed44378a73358db5fd0b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.635752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff0a86afaed44378a73358db5fd0b95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:02.650413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 874/1744] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:02 2022] DELETE /identity/v3/users/bff0a86afaed44378a73358db5fd0b95 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:03.070239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:03.070712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:03.070785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:03.196633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:03.198815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:03.337826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:03.377579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 871/1745] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:36:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:03.912407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:03.915233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:03.934551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:04.073949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42ff75-97da-4fa3-b3f6-0d12f5264231 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:04.080518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:04.080903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:04.081451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:04.081936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:04.091953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:04.094161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b52c-2b51-41df-8765-0acb1a7a770e service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:04.114466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 875/1746] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:36:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:36:04.306583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:04.306583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:04.563307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647010b-7841-43ed-8cd6-a44d5ecf3a08 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:04.572534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:04.573609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:04.574059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:04.575083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] PATH_INFO: `/v3/projects/9d5fdc640a7b4fd692408ea43be96dda` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:04.578564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5fdc640a7b4fd692408ea43be96dda)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:04.581791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:04.836362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5fdc640a7b4fd692408ea43be96dda', 'request_id': 'req-282edf0e-f0a7-4923-9610-cc61ad26f9da'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:04.913882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5fdc640a7b4fd692408ea43be96dda', 'request_id': 'req-282edf0e-f0a7-4923-9610-cc61ad26f9da'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:04.924683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5fdc640a7b4fd692408ea43be96dda', 'request_id': 'req-282edf0e-f0a7-4923-9610-cc61ad26f9da'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:04.993137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] The token cache is being invalidate because project 9d5fdc640a7b4fd692408ea43be96dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:04.993728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:04.997116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-282edf0e-f0a7-4923-9610-cc61ad26f9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:05.002743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 872/1747] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:04 2022] DELETE /identity/v3/projects/9d5fdc640a7b4fd692408ea43be96dda => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:05.008467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.009127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.009401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.077190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:05.082162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:05.247782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 876/1748] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:05.256248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.256719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.257200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.296478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:05.298062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:05.324209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:05.324603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 873/1749] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:05.342803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e3e2c-edff-46d8-8ee3-67401945f541 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:05.354465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:05.357114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.357513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.359081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.360936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:05.361416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:05.367853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:05.398825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 877/1750] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:36:05.403460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:05.415969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d64e7-16b6-4aaf-910a-f19cf089aa07 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:05.445666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:05.457152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.457152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.457152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.457152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:05.457152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:05.533368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:05.535358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:05.616822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e07a-e51c-4381-b3c8-444af88af4a9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:05.616822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:05.616822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.616822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.617593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] PATH_INFO: `/v3/users/1ea6ddc49e7f411dad9408789c4cb8b6` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.617593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea6ddc49e7f411dad9408789c4cb8b6)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:05.617593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:05.732030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 874/1751] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:05 2022] POST /identity/v3/projects => generated 367 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:05.736121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:05.910708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511caa61-0515-41c9-9bf0-d9a2415986d8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:05.924528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea6ddc49e7f411dad9408789c4cb8b6'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:05.925814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:05.927188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:05.927488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:05.933291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:05.935712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:05.938948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:06.011906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea6ddc49e7f411dad9408789c4cb8b6'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:06.041831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea6ddc49e7f411dad9408789c4cb8b6'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:06.045945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea6ddc49e7f411dad9408789c4cb8b6'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:06.138878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 875/1752] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:05 2022] POST /identity/v3/users => generated 284 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:06.159282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:06.210687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-97bbc542-c32a-4527-81be-85426944f9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea6ddc49e7f411dad9408789c4cb8b6'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:06.235614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 878/1753] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:05 2022] DELETE /identity/v3/users/1ea6ddc49e7f411dad9408789c4cb8b6 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:06.249780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae50c0-7bd0-4656-b3f8-ba5bc12e3aff None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.251235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae50c0-7bd0-4656-b3f8-ba5bc12e3aff None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.252342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae50c0-7bd0-4656-b3f8-ba5bc12e3aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.323874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9760d7-85c3-4ee3-b1a1-96ed10269c9e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:06.414819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d6ae50c0-7bd0-4656-b3f8-ba5bc12e3aff None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:06.418039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d6ae50c0-7bd0-4656-b3f8-ba5bc12e3aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:06.547381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:06.552115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.552482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:06.553231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.554010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.554933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:06.557118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:06.585697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 876/1754] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:06.591028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:06.593240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:06.602870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 879/1755] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:06.624716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.625026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.625353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.632012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ed474-e99c-43ab-9f9e-48f70109fd88 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:06.643730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:06.645854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.648256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.648776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.669280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:06.679501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:06.685086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:06.688625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:06.729606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 877/1756] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:06.734298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:06.739362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:06.895203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f75e-4ab9-4f68-a66d-3be348d08fe5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:06.900095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:06.900937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.901354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.901753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.908204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 880/1757] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Jul 29 14:36:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:06.911934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:06.943674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab365647-bbae-49ed-b1ee-5d66cea10d37 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:06.953454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:06.953454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:06.953454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:06.953454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] PATH_INFO: `/v3/projects/0d8d1cbfbd3c4565b1397a0f9abeb505/users/36330ff264e94d9da0773bfb91624652/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:06.963115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:06.965178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:06.988050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 878/1758] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:06.990025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:07.008600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b68753-14cf-4dbc-b3b5-2bd4df4478a5 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:07.011340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8d1cbfbd3c4565b1397a0f9abeb505, user_id=36330ff264e94d9da0773bfb91624652, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:07.015311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:07.023567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:07.030493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:07.030493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:07.030493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:07.055486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:07.057895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:07.080740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 879/1759] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:07.082986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:07.092442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4250-ba90-40f8-b38f-3d7759a7c416 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:07.098266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:07.116443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:07.116845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:07.117326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:07.127142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:07.128910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d88c5-6848-42b5-a8f5-77852a9751f5 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:07.142505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 880/1760] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:07.336795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 881/1761] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:06 2022] PUT /identity/v3/projects/0d8d1cbfbd3c4565b1397a0f9abeb505/users/36330ff264e94d9da0773bfb91624652/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:07.347577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:07.348119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:07.348566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:07.406814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] MFA Rules not processed for user `36330ff264e94d9da0773bfb91624652`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:07.408344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:07.447096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:07.549438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 881/1762] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:07.550565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:07.737197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a3fc-70f7-49d7-ac0d-4d33a74f6b01 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:07.762966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:07.763679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:07.763972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:07.764253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:07.785225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:07.789077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:07.951876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 882/1763] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:07 2022] POST /identity/v3/projects => generated 369 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:07.957009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:08.011544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b418e0-2b35-42d9-8317-a71517c32621 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:08.185012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:08.186051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:08.187331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:08.188398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:08.189592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:08.193095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:08.327038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 882/1764] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:07 2022] POST /identity/v3/users => generated 291 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:08.331524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:08.351659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8265ead-ecd2-4d44-ac03-027c4c24aa72 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:08.360718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:08.361652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:08.361937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:08.362368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:08.363244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:08.364925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:08.399143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:08.413479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 883/1765] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:08.420415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af779e7-8bc8-4b9e-bd60-99f4bf828c91 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:08.426341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:08.427896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:08.428657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:08.445246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] PATH_INFO: `/v3/projects/e5beaa7d84b647ea917ee766f0106345/users/9f16ab230928458abd1791403257ca5d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:08.472457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5beaa7d84b647ea917ee766f0106345, user_id=9f16ab230928458abd1791403257ca5d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:08.477108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:08.897150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:08.898652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 883/1766] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:08 2022] PUT /identity/v3/projects/e5beaa7d84b647ea917ee766f0106345/users/9f16ab230928458abd1791403257ca5d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:08.917413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75974d1-8d5a-48de-b4f2-4adb75003376 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:09.063954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:09.065342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:09.066545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:09.068309 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:09.070168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:09.080771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0becbe-e475-4479-9c32-f436b6a696d8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:09.105030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 884/1767] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:09.107495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:09.138116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e69e-a04b-41b0-97d9-940050890f89 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:09.141270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:09.165814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:09.166525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:09.166845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:09.167160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] PATH_INFO: `/v3/projects/e5beaa7d84b647ea917ee766f0106345/users/9f16ab230928458abd1791403257ca5d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:09.173874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5beaa7d84b647ea917ee766f0106345, user_id=9f16ab230928458abd1791403257ca5d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:09.178618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:09.189853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:09.237879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 884/1768] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:09 2022] PUT /identity/v3/projects/e5beaa7d84b647ea917ee766f0106345/users/9f16ab230928458abd1791403257ca5d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:09.252032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:09.252325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:09.252604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:09.314716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] MFA Rules not processed for user `9f16ab230928458abd1791403257ca5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:09.316019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:09.477844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 885/1769] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:09 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:09.486995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:09.489360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:09.786720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a33054-92f9-4145-90e2-337c9197b62e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:09.797754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:09.805018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:09.805018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:09.805018 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:09.825954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:09.831808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:09.880221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 885/1770] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:09 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:10.975680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:10.977179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:11.122838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:11.138376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82e675-801f-45a1-965b-ecbdf14131e9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:11.143480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:11.147676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:11.147676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:11.147676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] PATH_INFO: `/v3/projects/6eed875a70e9411785d28bdbf7e8bc55` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:11.147676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eed875a70e9411785d28bdbf7e8bc55)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:11.148661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:11.298355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eed875a70e9411785d28bdbf7e8bc55', 'request_id': 'req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:11.313720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eed875a70e9411785d28bdbf7e8bc55', 'request_id': 'req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:11.552413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eed875a70e9411785d28bdbf7e8bc55', 'request_id': 'req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:11.589385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] The token cache is being invalidate because project 6eed875a70e9411785d28bdbf7e8bc55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:11.590652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:11.591950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8c05559f-b175-4db2-b2e3-7cae1f19a8a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:11.599660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 886/1771] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:10 2022] DELETE /identity/v3/projects/6eed875a70e9411785d28bdbf7e8bc55 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:11.599916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:11.624868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fa4ce-6a2c-4746-a04d-1de5b1e7a8a2 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:11.688350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:11.763546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4976d-2566-4cc6-9e2d-4271a6dc882c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:11.763822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4976d-2566-4cc6-9e2d-4271a6dc882c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:11.764441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4976d-2566-4cc6-9e2d-4271a6dc882c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:11.766132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 887/1772] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 29 14:36:11.800448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:11.802230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:11.802769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:11.803286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] PATH_INFO: `/v3/users/e6a82817a0f048eab267eda75cd99c1c` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:11.804943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a82817a0f048eab267eda75cd99c1c)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:11.807678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:11.876338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:11.876947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:11.877460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:11.937153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:11.938645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:12.052283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 888/1773] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:12.136825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a82817a0f048eab267eda75cd99c1c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:12.178122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a82817a0f048eab267eda75cd99c1c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:12.290127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a82817a0f048eab267eda75cd99c1c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:12.304652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a82817a0f048eab267eda75cd99c1c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:12.308041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-befc1b6a-4cee-4aca-8c1d-0878a997500b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a82817a0f048eab267eda75cd99c1c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:12.330416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 886/1774] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:11 2022] DELETE /identity/v3/users/e6a82817a0f048eab267eda75cd99c1c => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:12.384545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:12.384545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:12.411226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4049b50-edd3-4cb0-ba7e-309c5b482d56 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:12.530636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:12.537209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:12.537209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:12.537209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:12.537209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:12.576834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:12.580881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d1c19-e405-4ca0-9d4a-08b73fce4a43 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:12.631880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 889/1775] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:13.395730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:13.396786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:13.397306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:13.401211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 887/1776] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:36:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:36:13.440775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:13.441812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:13.442400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:13.524538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:13.524538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:13.554890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 890/1777] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:14.022980 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:14.665499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:14.668512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:14.717554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc5443-5e2c-4fc0-80ed-02132828b259 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:14.732821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:14.733923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:14.734462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:14.735155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:14.749185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:14.752033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:14.790872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 888/1778] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:15.519120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:15.521493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:15.697915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02807852-e8e9-47b7-8041-25e1e78b5722 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:15.704874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:15.705875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:15.706365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:15.706850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] PATH_INFO: `/v3/projects/2e6c6537708a42fda922893df4f61036` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:15.711775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6c6537708a42fda922893df4f61036)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:15.714342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:15.890135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:15.895660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:15.904090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6c6537708a42fda922893df4f61036', 'request_id': 'req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:15.918405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41977c17-12a6-4a17-b710-2ae4e65b659d service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:15.933748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6c6537708a42fda922893df4f61036', 'request_id': 'req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:15.997935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6c6537708a42fda922893df4f61036', 'request_id': 'req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.015379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] The token cache is being invalidate because project 2e6c6537708a42fda922893df4f61036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:16.016080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.018528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fcf8601c-3e0c-4fef-bca0-cb63c8ecf7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.022652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 891/1779] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:15 2022] DELETE /identity/v3/projects/2e6c6537708a42fda922893df4f61036 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:16.027454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:16.027778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5a0e5-0fe0-4896-827c-318364d458c0 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.028236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5a0e5-0fe0-4896-827c-318364d458c0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.028677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5a0e5-0fe0-4896-827c-318364d458c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.028904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.029404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.029886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] PATH_INFO: `/v3/users/36330ff264e94d9da0773bfb91624652` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.032428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] RBAC: Authorizing `identity:delete_user(user_id=36330ff264e94d9da0773bfb91624652)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:16.034987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:16.174836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5ed5a0e5-0fe0-4896-827c-318364d458c0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:16.176672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5ed5a0e5-0fe0-4896-827c-318364d458c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:16.187363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 892/1780] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:16.198377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.199038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.200070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.266614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:16.271081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:16.294310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 893/1781] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:16.295320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:16.316587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e7dd0-9b9d-4ae9-b7e0-c826d2ece704 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:16.333592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:16.334251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.334836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.335263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.336190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:16.336287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:16.348312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:16.378607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 894/1782] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:16.383344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:16.403100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b690b-1c06-4635-ae4e-9a108b52a187 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:16.411919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:16.412564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.412872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.413163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.414374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:16.415766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:16.597793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:16.635653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 895/1783] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:16 2022] POST /identity/v3/projects => generated 381 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:16.648956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:16.697884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36330ff264e94d9da0773bfb91624652'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.719659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36330ff264e94d9da0773bfb91624652'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.811171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36330ff264e94d9da0773bfb91624652'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.824812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1e2c6-4f79-4832-b9ac-64fcad5eda49 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:16.828877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36330ff264e94d9da0773bfb91624652'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.831606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c2c5c03-f9cf-4208-9423-77ea9f3592da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36330ff264e94d9da0773bfb91624652'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:16.853826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:16.854815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.855251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.855684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.855817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 889/1784] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:15 2022] DELETE /identity/v3/users/36330ff264e94d9da0773bfb91624652 => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:16.857408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:16.860747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:16.861359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:16.861816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:16.862412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:16.903269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:16.905564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:16.998587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 896/1785] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:16 2022] POST /identity/v3/users => generated 291 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:17.002996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.005612 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:17.058256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283609f5-32e1-4c4e-a101-52f82b0f4253 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.078061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 890/1786] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:17.085892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.089064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.089064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.089064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.089064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:17.095591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:17.119199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 897/1787] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:17.123014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.135279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe3069-705d-4994-a9fc-6d02d86d94d0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.146959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.147749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.148118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.148313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.149261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:17.156224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:17.179513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.186590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 891/1788] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:17.190588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055c7cf-37fe-4986-a4f1-cefe802978ac admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.199329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.200614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.201138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.201609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] PATH_INFO: `/v3/projects/80de3094d4de432ab1193622e043babf/users/d4fd48a280a448ad8a1a2ef7f0080f29/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.222529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.224492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:17.265599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80de3094d4de432ab1193622e043babf, user_id=d4fd48a280a448ad8a1a2ef7f0080f29, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:17.270771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a0bd11-bcc6-4c92-8495-33a00e5b5a58 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:17.292685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:17.365743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 898/1789] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:17 2022] PUT /identity/v3/projects/80de3094d4de432ab1193622e043babf/users/d4fd48a280a448ad8a1a2ef7f0080f29/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:17.367224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.371147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5e3c2-2580-4124-84ef-bafbf1adba96 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.372482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.372897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.419505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] MFA Rules not processed for user `d4fd48a280a448ad8a1a2ef7f0080f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:17.420813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:17.483567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.484392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.511511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.511511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.579398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 899/1790] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:17.584144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.678172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:17.685918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:17.728455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 892/1791] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:17.755230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b61fc4-2ed0-4b2d-909b-531f802b7de5 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.762285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.763057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.763716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.764294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.765102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:17.771455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:17.901300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 900/1792] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:17 2022] POST /identity/v3/projects => generated 381 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:17.915505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:17.927137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5435d-47b2-4a16-94f1-831916e5653c service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:17.996088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:17.996945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:17.997501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:17.997948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:17.998886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.002101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.115968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 893/1793] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:17 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:18.117280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.130890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebef0a-442e-4c2b-b103-89df21df77c7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.135106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.135850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.136317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.136491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.137336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.139016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.168154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 901/1794] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:18.169307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.186824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830aa1-3bca-4798-9d5c-cc70f0c0b676 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.192584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.193600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.194232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.194568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] PATH_INFO: `/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9/users/63632b0b31a54d089938fbb834eb448d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.212962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23eb511e00974a8a9d87526bd6e6b8e9, user_id=63632b0b31a54d089938fbb834eb448d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.215758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.328087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 894/1795] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:18 2022] PUT /identity/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9/users/63632b0b31a54d089938fbb834eb448d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:18.331342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.342668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b7f0-8e8e-4bac-8012-3980ba7e45b1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.401714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.402385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.402679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.402950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.403636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.405262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.415220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 902/1796] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:18.419554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.430177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667dc8d-4123-44b2-a322-931840388d02 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.435502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.436420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.436786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.437199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] PATH_INFO: `/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9/users/63632b0b31a54d089938fbb834eb448d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.439321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=23eb511e00974a8a9d87526bd6e6b8e9, user_id=63632b0b31a54d089938fbb834eb448d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.443240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c318f-97b1-46ec-84ef-7d36761370ef admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.452275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.454506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 895/1797] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:18 2022] PUT /identity/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9/users/63632b0b31a54d089938fbb834eb448d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:18.454855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:18.467189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.467554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.468045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.470244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ac192-470a-4f4b-88d3-8df59bd4bc4f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.502600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] MFA Rules not processed for user `63632b0b31a54d089938fbb834eb448d`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:18.504231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:18.539603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.540266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.540549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.540836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] PATH_INFO: `/v3/projects/0d8d1cbfbd3c4565b1397a0f9abeb505` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.542060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8d1cbfbd3c4565b1397a0f9abeb505)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.543983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.586963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 896/1798] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:18.597408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.598981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:18.644268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8d1cbfbd3c4565b1397a0f9abeb505', 'request_id': 'req-2b0ca4df-135a-4247-866d-9cc80418b45c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:18.651131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8d1cbfbd3c4565b1397a0f9abeb505', 'request_id': 'req-2b0ca4df-135a-4247-866d-9cc80418b45c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:18.666267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eefae-8039-4780-8460-4b006e70c00e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.693047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:18.717356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.717982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.718263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.718538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.738059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8d1cbfbd3c4565b1397a0f9abeb505', 'request_id': 'req-2b0ca4df-135a-4247-866d-9cc80418b45c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:18.752189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] The token cache is being invalidate because project 0d8d1cbfbd3c4565b1397a0f9abeb505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:18.752479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:18.753193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:18.755850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 903/1799] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:18 2022] DELETE /identity/v3/projects/0d8d1cbfbd3c4565b1397a0f9abeb505 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:18.760027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:18.761595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:18.794462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.796358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:18.809074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 897/1800] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:18.837292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca4df-135a-4247-866d-9cc80418b45c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:18.842333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:18.848670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:18.848670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:18.848670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] PATH_INFO: `/v3/users/9f16ab230928458abd1791403257ca5d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:18.848670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f16ab230928458abd1791403257ca5d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:18.848670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:19.065773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:19.072139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f16ab230928458abd1791403257ca5d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.085358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f16ab230928458abd1791403257ca5d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.099109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f16ab230928458abd1791403257ca5d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.100040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f16ab230928458abd1791403257ca5d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.178593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f16ab230928458abd1791403257ca5d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.185637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 904/1801] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:18 2022] DELETE /identity/v3/users/9f16ab230928458abd1791403257ca5d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:19.405660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:19.416766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccde00-e347-417d-a09d-d22513117108 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:19.483519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:19.484279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:19.484657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:19.485171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] PATH_INFO: `/v3/users/d4fd48a280a448ad8a1a2ef7f0080f29` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:19.490169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fd48a280a448ad8a1a2ef7f0080f29)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:19.490169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:19.685994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:19.687704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:19.705261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fd48a280a448ad8a1a2ef7f0080f29'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.723348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fd48a280a448ad8a1a2ef7f0080f29'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.723447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a0390-9a29-4bba-8d72-0087a95edcae admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:19.729574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:19.730126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:19.730534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:19.730902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] PATH_INFO: `/v3/projects/e5beaa7d84b647ea917ee766f0106345` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:19.732177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5beaa7d84b647ea917ee766f0106345)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:19.734361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:19.787442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fd48a280a448ad8a1a2ef7f0080f29'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.798006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fd48a280a448ad8a1a2ef7f0080f29'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.798810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-581c2b66-dc82-47f3-80f0-0def02464b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fd48a280a448ad8a1a2ef7f0080f29'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.806520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 898/1802] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:19 2022] DELETE /identity/v3/users/d4fd48a280a448ad8a1a2ef7f0080f29 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:19.811186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:19.811538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:19.811826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:19.836226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:19.837497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:19.857747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5beaa7d84b647ea917ee766f0106345', 'request_id': 'req-eab1cfec-7f95-462f-b6c1-73d3c644af62'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.883535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5beaa7d84b647ea917ee766f0106345', 'request_id': 'req-eab1cfec-7f95-462f-b6c1-73d3c644af62'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:19.909086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 899/1803] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:19.946748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:19.948306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:19.995420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5beaa7d84b647ea917ee766f0106345', 'request_id': 'req-eab1cfec-7f95-462f-b6c1-73d3c644af62'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.017921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] The token cache is being invalidate because project e5beaa7d84b647ea917ee766f0106345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:20.018181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.020982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-eab1cfec-7f95-462f-b6c1-73d3c644af62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.027493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 905/1804] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:19 2022] DELETE /identity/v3/projects/e5beaa7d84b647ea917ee766f0106345 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:20.032654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.032829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1401-3c06-4fe2-a804-761f8bd759db None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.033421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.033811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.040429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.041120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.041766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.042059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.044377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.063787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.069073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81c8d-3b35-4719-b983-6f8612621326 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.078404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 900/1805] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:20.104382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:20.105956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.137287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 906/1806] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:20.139448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.139845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.140225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.181659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:20.186736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:20.190035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.210358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 901/1807] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:20.214965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.229799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f5d9-13c2-4d05-9995-1f43cc530194 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.235053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.235896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.236438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.237186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.246586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:20.247521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.249501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.263346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 907/1808] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:20.268556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.282074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcca22-05e3-4e7d-98ea-ac8eb2ddef6b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.287576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.288496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.288780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.289090 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.289767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.293583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.404681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 902/1809] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/projects => generated 401 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:20.409724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:20.410290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.420488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e13f1-177a-4f22-b982-c8fed2b4c228 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.478786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.479526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.479921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.480216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.480943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.483162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.506033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.508007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.521657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880471a4-197a-4820-9260-f64b124f6920 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.526603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.527336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.527788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.528068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] PATH_INFO: `/v3/users/bdc8ea78e5c049ec8468e9326fa75783` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.529401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc8ea78e5c049ec8468e9326fa75783)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.531490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.605883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 908/1810] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/users => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:20.610599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.621274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa78fd-7867-472f-bdb6-db9caf2528b3 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.628378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.628378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.628378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.628378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.628893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.630817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.641909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 909/1811] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:20.646407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.656120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7fe3-9867-4716-83a2-fc83194609a1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.661239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.661963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.662347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.662692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.663460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.665139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.675151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 910/1812] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:20.679347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.690662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3018f-84c1-48cb-8c42-fbff3a75f555 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.696333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.697032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.697501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.697782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] PATH_INFO: `/v3/projects/b280698f3bbf4a89854182ce01c4bd84/users/15ee03f0ce294229b419c062377692c7/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.714290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b280698f3bbf4a89854182ce01c4bd84, user_id=15ee03f0ce294229b419c062377692c7, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.717185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c219-1d61-44d4-8dd5-a96821fdb55d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.782749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc8ea78e5c049ec8468e9326fa75783'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.794785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc8ea78e5c049ec8468e9326fa75783'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.797767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 911/1813] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:20 2022] PUT /identity/v3/projects/b280698f3bbf4a89854182ce01c4bd84/users/15ee03f0ce294229b419c062377692c7/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:20.808798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213271-3dbf-4036-8845-a73e8bde2a5a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.809124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213271-3dbf-4036-8845-a73e8bde2a5a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.809404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8213271-3dbf-4036-8845-a73e8bde2a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.837982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e8213271-3dbf-4036-8845-a73e8bde2a5a None None] MFA Rules not processed for user `15ee03f0ce294229b419c062377692c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:20.839237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e8213271-3dbf-4036-8845-a73e8bde2a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.850996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc8ea78e5c049ec8468e9326fa75783'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.859852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc8ea78e5c049ec8468e9326fa75783'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.861321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc8ea78e5c049ec8468e9326fa75783'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:20.869334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 903/1814] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:20 2022] DELETE /identity/v3/users/bdc8ea78e5c049ec8468e9326fa75783 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:20.870383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.871859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.907941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 912/1815] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:20.909234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.909521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.909797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.929448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:20.930657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.938353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 913/1816] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:20.939103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:20.939978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8f27-59c5-4cf9-9d26-03147c89f852 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.940288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:20.946320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.947017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.947301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.947589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] PATH_INFO: `/v3/projects/80de3094d4de432ab1193622e043babf` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.948850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80de3094d4de432ab1193622e043babf)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.950914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:20.953110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cd8c7-6078-41ac-bea1-b894ced31325 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:20.958211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:20.958830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:20.959109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:20.959383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:20.960045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:20.961966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.074521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80de3094d4de432ab1193622e043babf', 'request_id': 'req-89308ccc-c877-4554-9b6d-1ecd40e77157'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.076961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 914/1817] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:20 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:21.080844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.080844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:21.139430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a3d5c-9790-40be-b8d6-12b6c30312ad admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.143023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.143637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.144111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.144386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.174978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80de3094d4de432ab1193622e043babf', 'request_id': 'req-89308ccc-c877-4554-9b6d-1ecd40e77157'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.183247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80de3094d4de432ab1193622e043babf', 'request_id': 'req-89308ccc-c877-4554-9b6d-1ecd40e77157'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.199580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] The token cache is being invalidate because project 80de3094d4de432ab1193622e043babf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:21.199580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.199580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.199580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 904/1818] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:20 2022] DELETE /identity/v3/projects/80de3094d4de432ab1193622e043babf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:21.199580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.200012 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:21.214348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.215864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.226427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 915/1819] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:21.227328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.237253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12418-90f1-4cac-b4cd-90497be234ae service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.243533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.244190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.244520 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.245075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] PATH_INFO: `/v3/users/63632b0b31a54d089938fbb834eb448d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.246483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63632b0b31a54d089938fbb834eb448d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.248157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.269611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308ccc-c877-4554-9b6d-1ecd40e77157 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.277279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.277279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.277279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.277279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.277834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.278851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.307005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:21.359445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 905/1820] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:21 2022] POST /identity/v3/users => generated 307 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:21.362949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.373297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e1dbb-dd80-4290-b03c-13b6a98e69ca admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.377978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.378586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.378876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.379150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.380395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.382050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.393551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 906/1821] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:21.398155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.419181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b502-a0ba-47de-90f6-605b7357341d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.424065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.424912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.425046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.425320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] PATH_INFO: `/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021/users/111e22e0b9e0422abdc2fb85694fe02b/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.445455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63632b0b31a54d089938fbb834eb448d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.448279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e4e82ce9746f3afba8825a2cfb021, user_id=111e22e0b9e0422abdc2fb85694fe02b, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.451584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.461069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63632b0b31a54d089938fbb834eb448d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.468605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63632b0b31a54d089938fbb834eb448d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.471475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63632b0b31a54d089938fbb834eb448d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.544845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:21.579331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63632b0b31a54d089938fbb834eb448d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:21.585757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 907/1822] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:21 2022] PUT /identity/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021/users/111e22e0b9e0422abdc2fb85694fe02b/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:21.585908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 916/1823] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:21 2022] DELETE /identity/v3/users/63632b0b31a54d089938fbb834eb448d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:21.592657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.594348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:21.682023 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb354ca-ddf4-490b-a4d1-02329e8a1b5e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.747782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.748416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.748702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.748984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.749721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.751335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.761335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 908/1824] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:21.765452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.772575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:21.781937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85b43f-d795-48e1-8290-db3936bacfa6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.783641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.784320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.784752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.785210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] PATH_INFO: `/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021/users/111e22e0b9e0422abdc2fb85694fe02b/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.787823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e4e82ce9746f3afba8825a2cfb021, user_id=111e22e0b9e0422abdc2fb85694fe02b, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.790519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d5bb4-3d4e-430f-ad7d-5971599464a1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.804512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 917/1825] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:21 2022] PUT /identity/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021/users/111e22e0b9e0422abdc2fb85694fe02b/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:21.814788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.814788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:21.817575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.818402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.818826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.831185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde1065-3a6d-4123-bc65-decf3aed85eb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:21.869517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] MFA Rules not processed for user `111e22e0b9e0422abdc2fb85694fe02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:21.878022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:21.921898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:21.921898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:21.921898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:21.921898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] PATH_INFO: `/v3/projects/fe967d331dee4898953eae76d7911f5e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:21.922389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe967d331dee4898953eae76d7911f5e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:21.922638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:21.965734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 918/1826] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:21.978606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:21.981225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.045099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37974-f5e9-427c-9097-89891033e1c4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.048719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.049368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.049646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.049917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.062073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.063555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.075771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe967d331dee4898953eae76d7911f5e', 'request_id': 'req-89f97be4-3591-4479-a099-b13d20c85218'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.075977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 919/1827] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:22.163736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe967d331dee4898953eae76d7911f5e', 'request_id': 'req-89f97be4-3591-4479-a099-b13d20c85218'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.170469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe967d331dee4898953eae76d7911f5e', 'request_id': 'req-89f97be4-3591-4479-a099-b13d20c85218'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.181936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] The token cache is being invalidate because project fe967d331dee4898953eae76d7911f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:22.182221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.183108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.184635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 909/1828] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:21 2022] DELETE /identity/v3/projects/fe967d331dee4898953eae76d7911f5e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:22.188634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.201819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.204448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba05d4-d5d0-425f-8993-a0cf77abcc5c service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.210137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.274338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.274992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.277121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.277121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] PATH_INFO: `/v3/users/e8035e5fca0540cc8e15a8e76a4fe039` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.277121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8035e5fca0540cc8e15a8e76a4fe039)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.278958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.328306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f97be4-3591-4479-a099-b13d20c85218 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.334226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.334873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.335152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.335425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] PATH_INFO: `/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.336611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23eb511e00974a8a9d87526bd6e6b8e9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.338548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.474866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8035e5fca0540cc8e15a8e76a4fe039'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.475083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23eb511e00974a8a9d87526bd6e6b8e9', 'request_id': 'req-ee4f381f-8e3f-49ce-8363-6427aec446d7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.486956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8035e5fca0540cc8e15a8e76a4fe039'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.493727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8035e5fca0540cc8e15a8e76a4fe039'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.494428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8035e5fca0540cc8e15a8e76a4fe039'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.554746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23eb511e00974a8a9d87526bd6e6b8e9', 'request_id': 'req-ee4f381f-8e3f-49ce-8363-6427aec446d7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.561503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23eb511e00974a8a9d87526bd6e6b8e9', 'request_id': 'req-ee4f381f-8e3f-49ce-8363-6427aec446d7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.572165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] The token cache is being invalidate because project 23eb511e00974a8a9d87526bd6e6b8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:22.572457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.573162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.574816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 910/1829] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:22 2022] DELETE /identity/v3/projects/23eb511e00974a8a9d87526bd6e6b8e9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:22.576278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.577683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.606317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-00531faa-8e4f-471d-bc39-9e4dd3b1dff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8035e5fca0540cc8e15a8e76a4fe039'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:22.613284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 920/1830] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:22 2022] DELETE /identity/v3/users/e8035e5fca0540cc8e15a8e76a4fe039 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:22.614624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dca3b-f1d9-4c9e-b188-65456b9fb920 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.614913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dca3b-f1d9-4c9e-b188-65456b9fb920 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.615898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dca3b-f1d9-4c9e-b188-65456b9fb920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.641057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f381f-8e3f-49ce-8363-6427aec446d7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.648125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-244dca3b-f1d9-4c9e-b188-65456b9fb920 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:22.652048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-244dca3b-f1d9-4c9e-b188-65456b9fb920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.702516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.705582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.706030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.706470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.719224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.743981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 921/1831] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:22.758112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.758403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.758683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.783557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:22.784394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:22.797163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 922/1832] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:22.801356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.808001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d38b9-479c-420f-80cf-610375b12dc6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.813329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.814000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.814283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.814563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.815266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:22.815568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.817793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.833207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 923/1833] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:22.836866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.837686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.839615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.848388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e027e9-d5b6-4141-8ed3-662faa832808 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:22.849797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 911/1834] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:22.853716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:22.854602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:22.854899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:22.855329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:22.855998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:22.863733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:22.863733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:22.951838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 924/1835] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:22 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:22.963603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:22.977819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533043ae-8fce-461f-a228-9543c7fc47ff service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.013870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:23.074782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.075328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.075609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.075890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.076604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.078557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.172423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 912/1836] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:22 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:23.173701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.185111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3891-b122-4ea3-b6d2-50054d1cfa30 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.189766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.191540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.191540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.191540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.192368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.194091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.204771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 925/1837] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:23.208812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.219565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e7355-0901-42dc-915d-ec13421122ef admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.224388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.225074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.225388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.225665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.226584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.228080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.235503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.237441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.239694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 913/1838] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:23.243607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.254852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aac90-3245-410a-a7e8-59f8a5229def admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.255040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c02f01-3b50-4d35-8c02-8a292c5c983e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.259661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.259991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.260335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.260618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.260717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.260976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] PATH_INFO: `/v3/projects/616e24958e89424a8bb4ab458c24b3af/users/87025e242699449690c069a278947f71/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.261100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.261493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] PATH_INFO: `/v3/projects/845fb99b4ce548b0bce4fe5da7188702` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.262834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=845fb99b4ce548b0bce4fe5da7188702)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.264636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.277974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616e24958e89424a8bb4ab458c24b3af, user_id=87025e242699449690c069a278947f71, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.280711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49737fe7-4765-4d41-a107-697a9fee7ba2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.387643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 914/1839] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:23 2022] PUT /identity/v3/projects/616e24958e89424a8bb4ab458c24b3af/users/87025e242699449690c069a278947f71/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:23.398674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.399033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.399394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.404670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '845fb99b4ce548b0bce4fe5da7188702', 'request_id': 'req-8eba560c-8816-4728-8b4d-b13fb675f133'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:23.415491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '845fb99b4ce548b0bce4fe5da7188702', 'request_id': 'req-8eba560c-8816-4728-8b4d-b13fb675f133'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:23.434905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] MFA Rules not processed for user `87025e242699449690c069a278947f71`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:23.436179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.492244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '845fb99b4ce548b0bce4fe5da7188702', 'request_id': 'req-8eba560c-8816-4728-8b4d-b13fb675f133'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:23.504096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 915/1840] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:23.514901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.516308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.537906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] The token cache is being invalidate because project 845fb99b4ce548b0bce4fe5da7188702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:23.538262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:23.540384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8eba560c-8816-4728-8b4d-b13fb675f133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:23.541986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 926/1841] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:23 2022] DELETE /identity/v3/projects/845fb99b4ce548b0bce4fe5da7188702 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:23.547758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.548043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.548446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.572738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:23.574092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.600217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0558be-c6e0-4dd0-b9bc-01815e132a2f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.604047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.604674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.604946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.605264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.606971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.619474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.621026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457cf39-5b25-4733-954e-9b8609b3f414 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.631014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 916/1842] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:23.640811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 927/1843] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:23.657231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.657231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.657231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.671578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:23.673072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:23.683305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 917/1844] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:23.687381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.698062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23885933-b9d1-404b-82da-8faf54ad8157 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.702637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.703270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.703536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.703813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.704510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:23.704812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.706468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.719043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 928/1845] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:23.724720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.734397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17963a-67f7-486b-8d19-721b9af9660f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:23.739175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:23.739826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:23.740108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:23.740407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:23.741164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:23.743083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:23.769432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:23.859525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 918/1846] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:23 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:23.863608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:23.874683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964905d7-b7ec-458c-8801-4fff3b249002 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.010456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.010456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.010456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.010456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.010729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.011425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.112483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 929/1847] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:23 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:24.117619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.132654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427afaf-e7e4-4218-b33f-8c23a980bafd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.138318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.138937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.139215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.142265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.142265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.142595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.155907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 919/1848] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:24.159987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.177264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a7bd-e7ca-459f-9eb3-89727c5ede57 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.184024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.184655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.184934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.185709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.186419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.188522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0caad7-7bae-4394-84f2-c84f65004d4b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.204635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 930/1849] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:24.209558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.220138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ae16-9be3-482e-8bad-fc208bfcce26 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.226566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.227246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.227577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.228197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] PATH_INFO: `/v3/projects/be806ec637784264a72701e4942ba937/users/f7077c80bd1a4e12a2d7eb7e5ff419c0/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.250050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be806ec637784264a72701e4942ba937, user_id=f7077c80bd1a4e12a2d7eb7e5ff419c0, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.252820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.384753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 920/1850] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:24 2022] PUT /identity/v3/projects/be806ec637784264a72701e4942ba937/users/f7077c80bd1a4e12a2d7eb7e5ff419c0/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:24.396003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.396244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.396523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.428165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] MFA Rules not processed for user `f7077c80bd1a4e12a2d7eb7e5ff419c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:24.429449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:24.495232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 931/1851] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:24.500004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.564101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf708c-4975-4736-8c36-f5708d0a0716 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.569851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.570536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.570852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.576522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.576522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.576522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.691423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:24.701519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 921/1852] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:24 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:24.705703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.719342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c9f89-80c9-40fe-9145-b22192486f15 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.793577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.794307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.794661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.795115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.795800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.797779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.889929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 932/1853] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:24 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:24.893985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.904608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc976681-d943-4825-9b28-6b80a736ec40 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.909515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.910139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.910425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.911413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.912129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.913775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:24.924684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 922/1854] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:24.930514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:24.941089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92b4c-d612-4067-bfff-7a980062091f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:24.947026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:24.947713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:24.948191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:24.948629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] PATH_INFO: `/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae/users/8f5a8351760740fda17777da5d57def7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:24.965796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cb3edb0d4648c0b4ea3e2aa01714ae, user_id=8f5a8351760740fda17777da5d57def7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:24.968640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:25.092232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 933/1855] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:24 2022] PUT /identity/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae/users/8f5a8351760740fda17777da5d57def7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:25.096162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:25.109263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e8869-43a8-442a-8081-195541d26ad3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:25.180175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:25.180851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.181126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.181382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.182279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:25.184316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67961684-dc98-4737-987b-705806b8a0d9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:25.199623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 923/1856] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:25.200154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:25.211304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eb3ad-08ba-48ab-bd7e-d9a4c9560aa5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:25.215849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:25.216510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.216790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.217087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] PATH_INFO: `/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae/users/8f5a8351760740fda17777da5d57def7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.219060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cb3edb0d4648c0b4ea3e2aa01714ae, user_id=8f5a8351760740fda17777da5d57def7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:25.221744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:25.236949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 934/1857] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:25 2022] PUT /identity/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae/users/8f5a8351760740fda17777da5d57def7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:25.248301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.248598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.248876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.293430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] MFA Rules not processed for user `8f5a8351760740fda17777da5d57def7`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:25.294635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:25.309974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:25.384493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 924/1858] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:25.395699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:25.397448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:25.459464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51783f-9c31-47cc-8096-e92cd946001f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:25.463024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:25.463658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.464076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.464248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.487357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:25.487357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d3e4a-d51a-477e-88b5-902ca8be7648 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:25.487993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 935/1859] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:25.517751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:25.591202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85c17b-bff5-4f8b-8c91-deb3953a2d64 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:25.595911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:25.596664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.597033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.597357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] PATH_INFO: `/v3/users/f7077c80bd1a4e12a2d7eb7e5ff419c0` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.599093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7077c80bd1a4e12a2d7eb7e5ff419c0)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:25.601016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:25.832269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7077c80bd1a4e12a2d7eb7e5ff419c0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:25.846917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7077c80bd1a4e12a2d7eb7e5ff419c0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:25.855050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:25.938848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7077c80bd1a4e12a2d7eb7e5ff419c0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:25.948617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7077c80bd1a4e12a2d7eb7e5ff419c0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:25.949555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7077c80bd1a4e12a2d7eb7e5ff419c0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:25.957149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 925/1860] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:25 2022] DELETE /identity/v3/users/f7077c80bd1a4e12a2d7eb7e5ff419c0 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:25.962693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2dd9a-cbd6-485c-a271-65ff305ade99 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:25.962986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2dd9a-cbd6-485c-a271-65ff305ade99 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:25.963267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2dd9a-cbd6-485c-a271-65ff305ade99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:25.989227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e7b2dd9a-cbd6-485c-a271-65ff305ade99 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:25.990762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e7b2dd9a-cbd6-485c-a271-65ff305ade99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:26.063408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 936/1861] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:26.064809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:26.075763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:26.083582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:26.141712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66102de5-7009-4918-a2f2-c698ce44ae0b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:26.146432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:26.147066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:26.147374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:26.147743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:26.169823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:26.171291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee10773-a6b9-4aa0-85b0-ba41b7b5fcae service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:26.183605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 926/1862] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:26.392229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:26.392759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:26.392759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:26.394591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 937/1863] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:36:26.411616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:26.412232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:26.412712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:26.477205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:26.478529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:26.587019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 927/1864] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:26.679484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:26.681495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:26.702379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fd028-b61b-4369-82f4-ac70840b395c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:26.708690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:26.709544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:26.709823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:26.710094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:26.719199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:26.720799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:26.733319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:26.733769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 938/1865] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:26.733847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:26.747888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd99651-2121-477c-97a9-33360cf3d571 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:26.752816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:26.753617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:26.753970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:26.754433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] PATH_INFO: `/v3/projects/be806ec637784264a72701e4942ba937` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:26.756648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be806ec637784264a72701e4942ba937)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:26.758996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:26.958772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be806ec637784264a72701e4942ba937', 'request_id': 'req-f8614b30-4364-4b18-97f0-f604bf16fad2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.020871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be806ec637784264a72701e4942ba937', 'request_id': 'req-f8614b30-4364-4b18-97f0-f604bf16fad2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.027802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be806ec637784264a72701e4942ba937', 'request_id': 'req-f8614b30-4364-4b18-97f0-f604bf16fad2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.038742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] The token cache is being invalidate because project be806ec637784264a72701e4942ba937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:27.039051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.039718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.041318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 928/1866] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:26 2022] DELETE /identity/v3/projects/be806ec637784264a72701e4942ba937 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:27.045567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:27.056839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9752d-5295-4558-a9f0-4be222914479 service nova] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:27.126431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:27.127086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:27.127368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:27.127646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] PATH_INFO: `/v3/users/8f5a8351760740fda17777da5d57def7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:27.128816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5a8351760740fda17777da5d57def7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:27.130820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:27.314489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5a8351760740fda17777da5d57def7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.326467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5a8351760740fda17777da5d57def7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.333448 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5a8351760740fda17777da5d57def7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.334124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5a8351760740fda17777da5d57def7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.394600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-227e47a7-0c99-48cc-9801-d09b34379ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5a8351760740fda17777da5d57def7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:27.400745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 939/1867] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:27 2022] DELETE /identity/v3/users/8f5a8351760740fda17777da5d57def7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:27.654405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:27.889409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:27.890985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:27.904579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8614b30-4364-4b18-97f0-f604bf16fad2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:27.924302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:27.978628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:27.978628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:27.978628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:27.978628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] PATH_INFO: `/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:27.979026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cb3edb0d4648c0b4ea3e2aa01714ae)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:27.979742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.090225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cb3edb0d4648c0b4ea3e2aa01714ae', 'request_id': 'req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:28.163804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cb3edb0d4648c0b4ea3e2aa01714ae', 'request_id': 'req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:28.171097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cb3edb0d4648c0b4ea3e2aa01714ae', 'request_id': 'req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:28.182495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] The token cache is being invalidate because project 37cb3edb0d4648c0b4ea3e2aa01714ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:28.184266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:28.184266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eca4fbcb-b22f-452e-9939-eaf6eb2151d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:28.191494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 929/1868] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:27 2022] DELETE /identity/v3/projects/37cb3edb0d4648c0b4ea3e2aa01714ae => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:28.193443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.193790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.194130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.214912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:28.216264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:28.278991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 940/1869] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:28.288135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.288420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.288701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.309093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:28.310340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:28.319662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 930/1870] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:28.324089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.334061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d219e-1e2d-4767-9acb-84addc6aa007 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.338716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.339323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.339596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.339864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.340550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:28.340851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.342552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.354009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 941/1871] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:28.358250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.368005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4036-bc45-402f-8373-deae0c373dd4 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.373082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.373683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.373962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.374238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.374901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.376817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.458790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 931/1872] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:28.463033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.472755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea68408-83d3-4e1a-b58a-ef9335ea34d8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.528939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.529575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.529853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.530133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.530899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.532784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.555149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:28.611658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 942/1873] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:28.616724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.626942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8a2fc-0620-4aeb-b37d-5ef19540f100 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.631772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.632396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.632724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.633016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.633761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.635401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.645334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 932/1874] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:28.649273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.659271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477bb2-48fc-491f-97d5-31397a3a4232 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.665097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.665716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.665996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.666267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.666963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.668534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb992133-5cf0-4d8e-a35e-4aac4b1b10ff admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.680198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 943/1875] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:28.684663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:28.695254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4341a98-292e-415c-b9b8-f3508a138a93 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:28.700725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:28.701456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.701978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.702113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] PATH_INFO: `/v3/projects/007e4b5a54b54c2f9b718b33ce153721/users/b857e4c3adfa4149ae17562d30f47150/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.705633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:28.719536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=007e4b5a54b54c2f9b718b33ce153721, user_id=b857e4c3adfa4149ae17562d30f47150, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:28.722196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:28.791937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 933/1876] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:28 2022] PUT /identity/v3/projects/007e4b5a54b54c2f9b718b33ce153721/users/b857e4c3adfa4149ae17562d30f47150/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:28.803890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:28.804467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:28.804930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:28.849398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] MFA Rules not processed for user `b857e4c3adfa4149ae17562d30f47150`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:28.852541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:28.931308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 944/1877] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:28.933008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.015531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4437-27de-44ac-ad5e-aadb7b268f1a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.019673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:29.026460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.029134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.029134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.029134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.029134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.031507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.092916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:29.173111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 934/1878] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:28 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:29.179704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.190765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817e39-9c39-4df8-aec7-92e83287ef55 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.245810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.246349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.246626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.246902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.247563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.249492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.353420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 945/1879] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:29 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:29.354614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.365769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8d7f0-ce89-4d79-b5f3-55d2f84fef5d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.370506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.371172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.371456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.371740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.372436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.374094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.388765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 935/1880] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:29.393372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.403895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c233-06f8-4b1a-8c04-cb5b8836c9d6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.408434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.409126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.409409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.409682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] PATH_INFO: `/v3/projects/4cd695e199564c5d95872dd6f7f39bbe/users/5f46aa4554644e879bfbb464c20c0c2e/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.425518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd695e199564c5d95872dd6f7f39bbe, user_id=5f46aa4554644e879bfbb464c20c0c2e, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.429443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.493330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 946/1881] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:29 2022] PUT /identity/v3/projects/4cd695e199564c5d95872dd6f7f39bbe/users/5f46aa4554644e879bfbb464c20c0c2e/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:29.497913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.508228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456108ec-5e69-40fc-a343-4b92ce234072 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.565337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.565949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.566227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.566496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.567531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.569237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a31ff-43e1-4fd5-b18a-10ed9274d307 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.578702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 936/1882] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:29.583204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.593658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d44f4-d811-4f64-a233-30e35474972d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.599396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.600055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.600333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.600607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] PATH_INFO: `/v3/projects/4cd695e199564c5d95872dd6f7f39bbe/users/5f46aa4554644e879bfbb464c20c0c2e/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.602576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd695e199564c5d95872dd6f7f39bbe, user_id=5f46aa4554644e879bfbb464c20c0c2e, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.605292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.616558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 947/1883] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:29 2022] PUT /identity/v3/projects/4cd695e199564c5d95872dd6f7f39bbe/users/5f46aa4554644e879bfbb464c20c0c2e/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:29.627585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.627872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.628148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.655670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] MFA Rules not processed for user `5f46aa4554644e879bfbb464c20c0c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:29.656883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:29.705100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:29.726427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 937/1884] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:29.738667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:29.740343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:29.797864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45c358-5062-42d7-b625-6545715e65ab admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:29.801717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:29.802324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:29.802703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:29.802985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:29.812285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:29.813724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:29.823228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 948/1885] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:29.829428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:31.075953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:31.282502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:31.298940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b642731-3864-42b9-9768-c9ef8b02481b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:31.313855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f46aa4554644e879bfbb464c20c0c2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd695e199564c5d95872dd6f7f39bbe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:31.314526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:31.315001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:31.315439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:31.316319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:31.316900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:31.318895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:31.349855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 938/1886] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:31.350919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:31.361840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e1eaf-d738-48ad-974b-9ba7b609c393 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:31.381469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f46aa4554644e879bfbb464c20c0c2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd695e199564c5d95872dd6f7f39bbe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:31.382111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:31.382397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:31.382681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:31.383376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:31.385383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:31.522247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 949/1887] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:31 2022] POST /identity/v3/projects => generated 355 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:31.524976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:31.536109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22377695-8491-439d-9e32-e01334268abe tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:31.609566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:31.611788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:31.644230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f46aa4554644e879bfbb464c20c0c2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd695e199564c5d95872dd6f7f39bbe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:31.645055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:31.645447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:31.645947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:31.647205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:31.647579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:31.649323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:31.666238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 939/1888] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:31.692174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:31.721860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a20738-a5ad-41aa-a296-8229459aff00 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:31.731433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:31.731433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:31.731433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:31.731433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] PATH_INFO: `/v3/users/15ee03f0ce294229b419c062377692c7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:31.731975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ee03f0ce294229b419c062377692c7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:31.738438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:31.741146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:31.752109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273a0-4286-4ceb-bd2c-4b2a7c01c2a7 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:31.757846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f46aa4554644e879bfbb464c20c0c2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd695e199564c5d95872dd6f7f39bbe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:31.758525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:31.758913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:31.759088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] PATH_INFO: `/v3/projects/3e62f350f2394949be98d52eae3fff7e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:31.761146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3e62f350f2394949be98d52eae3fff7e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:31.763602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:31.873415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:32.008956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e62f350f2394949be98d52eae3fff7e', 'request_id': 'req-c21a60ff-69bf-475e-9c7f-29a581109250'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.011387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ee03f0ce294229b419c062377692c7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.030765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ee03f0ce294229b419c062377692c7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.041265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ee03f0ce294229b419c062377692c7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.042678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ee03f0ce294229b419c062377692c7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.141218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e62f350f2394949be98d52eae3fff7e', 'request_id': 'req-c21a60ff-69bf-475e-9c7f-29a581109250'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.153377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e62f350f2394949be98d52eae3fff7e', 'request_id': 'req-c21a60ff-69bf-475e-9c7f-29a581109250'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.168313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] The token cache is being invalidate because project 3e62f350f2394949be98d52eae3fff7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:32.168710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.169536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c21a60ff-69bf-475e-9c7f-29a581109250 tempest-VolumeQuotaClassesTest-815489773 tempest-VolumeQuotaClassesTest-815489773-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.171403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 940/1889] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:31 2022] DELETE /identity/v3/projects/3e62f350f2394949be98d52eae3fff7e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:32.204229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ee03f0ce294229b419c062377692c7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:32.212168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 950/1890] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:31 2022] DELETE /identity/v3/users/15ee03f0ce294229b419c062377692c7 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:32.216937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:32.217272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:32.217599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:32.238702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:32.239974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:32.305717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 941/1891] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:32.323668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:32.323668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:32.388045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60837e01-5f6b-4385-b905-9e6aa15ceb20 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:32.391910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:32.392603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:32.392974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:32.393282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:32.403412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:32.408095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:32.415527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 951/1892] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:32.444212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:32.951456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:32.959405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:32.962160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:32.963535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:32.972718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4734dcd-1533-4295-bb4f-7b91995dfe20 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:32.978713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:32.979647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:32.979795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78bea1-e35f-46c2-bbeb-c17af0fa82ac service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:32.979983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:32.980458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] PATH_INFO: `/v3/users/b857e4c3adfa4149ae17562d30f47150` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:32.981731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b857e4c3adfa4149ae17562d30f47150)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:32.983773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:32.984437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:32.985658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:32.986234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:32.986674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] PATH_INFO: `/v3/projects/b280698f3bbf4a89854182ce01c4bd84` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:32.988246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b280698f3bbf4a89854182ce01c4bd84)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:32.990679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:33.177951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b280698f3bbf4a89854182ce01c4bd84', 'request_id': 'req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.182586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b857e4c3adfa4149ae17562d30f47150'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.185603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:33.186381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b280698f3bbf4a89854182ce01c4bd84', 'request_id': 'req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.198938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b857e4c3adfa4149ae17562d30f47150'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.278813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b280698f3bbf4a89854182ce01c4bd84', 'request_id': 'req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.291457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] The token cache is being invalidate because project b280698f3bbf4a89854182ce01c4bd84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:33.291742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.292356 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.293932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 952/1893] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:32 2022] DELETE /identity/v3/projects/b280698f3bbf4a89854182ce01c4bd84 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:33.297937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:33.299287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:33.330666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:33.362325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b5aa9-9c22-4813-a544-3ddd42e7acf7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:33.367678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:33.367841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:33.368127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:33.368402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] PATH_INFO: `/v3/users/111e22e0b9e0422abdc2fb85694fe02b` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:33.369587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111e22e0b9e0422abdc2fb85694fe02b)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:33.371504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:33.386059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b857e4c3adfa4149ae17562d30f47150'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.393696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b857e4c3adfa4149ae17562d30f47150'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.394397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d422308f-6ea9-453b-862d-2763a4211dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b857e4c3adfa4149ae17562d30f47150'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.401471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 942/1894] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:32 2022] DELETE /identity/v3/users/b857e4c3adfa4149ae17562d30f47150 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:33.405965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:33.406263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:33.406655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:33.425191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:33.426964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:33.483706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 943/1895] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:33.494129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:33.495506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:33.550857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a87be-6008-491f-a642-beec07022f39 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:33.555136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:33.555760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:33.556042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:33.556320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:33.566591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:33.568394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:33.572176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111e22e0b9e0422abdc2fb85694fe02b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.580299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 944/1896] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:33.586265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111e22e0b9e0422abdc2fb85694fe02b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.594408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111e22e0b9e0422abdc2fb85694fe02b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.594836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111e22e0b9e0422abdc2fb85694fe02b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.686628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111e22e0b9e0422abdc2fb85694fe02b'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:33.696844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 953/1897] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:33 2022] DELETE /identity/v3/users/111e22e0b9e0422abdc2fb85694fe02b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:34.435423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:34.438514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:34.440218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:34.442781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:34.459420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341346c8-fc13-440f-9251-91ba64ab607d service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:34.530764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:34.531410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.531693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.531966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] PATH_INFO: `/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.533156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2e4e82ce9746f3afba8825a2cfb021)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:34.535176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:34.536465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba644f-737d-4e79-a2b7-9c8c8af02f16 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:34.541109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:34.543269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.543269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.543269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] PATH_INFO: `/v3/projects/007e4b5a54b54c2f9b718b33ce153721` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.543615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=007e4b5a54b54c2f9b718b33ce153721)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:34.545530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:34.547192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:34.660191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2e4e82ce9746f3afba8825a2cfb021', 'request_id': 'req-f4c03689-9e07-4146-af7d-c121740d3ab2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.672900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007e4b5a54b54c2f9b718b33ce153721', 'request_id': 'req-7c4b6e92-f572-493d-a779-a3cf02852346'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.674085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:34.680277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007e4b5a54b54c2f9b718b33ce153721', 'request_id': 'req-7c4b6e92-f572-493d-a779-a3cf02852346'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.720499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2e4e82ce9746f3afba8825a2cfb021', 'request_id': 'req-f4c03689-9e07-4146-af7d-c121740d3ab2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.728126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2e4e82ce9746f3afba8825a2cfb021', 'request_id': 'req-f4c03689-9e07-4146-af7d-c121740d3ab2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.740984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] The token cache is being invalidate because project ed2e4e82ce9746f3afba8825a2cfb021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:34.741436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.742301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4c03689-9e07-4146-af7d-c121740d3ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.744641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 945/1898] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:34 2022] DELETE /identity/v3/projects/ed2e4e82ce9746f3afba8825a2cfb021 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:34.749237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349023d-ab55-4ae7-93aa-b4de2006b8ef None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.749519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349023d-ab55-4ae7-93aa-b4de2006b8ef None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.749788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349023d-ab55-4ae7-93aa-b4de2006b8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.769074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b349023d-ab55-4ae7-93aa-b4de2006b8ef None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:34.770730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b349023d-ab55-4ae7-93aa-b4de2006b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:34.788914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007e4b5a54b54c2f9b718b33ce153721', 'request_id': 'req-7c4b6e92-f572-493d-a779-a3cf02852346'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.800166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] The token cache is being invalidate because project 007e4b5a54b54c2f9b718b33ce153721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:34.800455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.801207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:34.803641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 954/1899] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:34 2022] DELETE /identity/v3/projects/007e4b5a54b54c2f9b718b33ce153721 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:34.807554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:34.809301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:34.831835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 946/1900] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:34.842823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.842823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.842823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.861991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:34.863222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:34.872742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 947/1901] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:34.877553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:34.883877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6e92-f572-493d-a779-a3cf02852346 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:34.887224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7280b-bf8f-4c35-8b9e-2938b3a5d4e9 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:34.889118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:34.889864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.890275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.890687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] PATH_INFO: `/v3/users/5f46aa4554644e879bfbb464c20c0c2e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.892142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f46aa4554644e879bfbb464c20c0c2e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:34.892338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:34.892946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.893270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.893550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.894260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:34.894570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:34.895265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:34.896220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:34.908548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 948/1902] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:34.913830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:34.935583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822009a8-ea7d-4e7a-8583-fd3fb462b5ab admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:34.940926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:34.941676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:34.942147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:34.942437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:34.943102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:34.946565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.058966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 949/1903] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:34 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:35.060080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.131764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692f75-416e-47e1-b1e1-9a9c5411a725 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.136632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.137298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.137584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.137865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.138532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.140658 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.144517 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f46aa4554644e879bfbb464c20c0c2e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.159978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f46aa4554644e879bfbb464c20c0c2e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.168604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f46aa4554644e879bfbb464c20c0c2e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.169575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f46aa4554644e879bfbb464c20c0c2e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.203665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:35.226219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f46aa4554644e879bfbb464c20c0c2e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.232082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 950/1904] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:35 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:35.232590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 955/1905] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:34 2022] DELETE /identity/v3/users/5f46aa4554644e879bfbb464c20c0c2e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:35.236368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.238009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.250918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39775f-b831-4fc0-acdc-3794990cebfa admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.310074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.310854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.311261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.311547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.313273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.315863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.326756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 951/1906] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:35.330605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.341947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97218a9-50dc-4802-a0a3-431511d97932 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.347579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.348537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.348889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.350882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.350882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.352132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.362629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.365031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.365148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 956/1907] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:35.368977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.378559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7afe-5d51-4cc3-9671-97f19869bcdb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.379868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c838f-5caf-417a-8005-da2f30f153b4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.384386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.385066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.385345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.385666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] PATH_INFO: `/v3/users/87025e242699449690c069a278947f71` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.386789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87025e242699449690c069a278947f71)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.386959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.387694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.388039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.388379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] PATH_INFO: `/v3/projects/dbf20b76ca594cfeb4076e4eae5cd1b3/users/eb14084591c84c348fedf9e003c4c49a/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.388851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.406806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf20b76ca594cfeb4076e4eae5cd1b3, user_id=eb14084591c84c348fedf9e003c4c49a, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.410582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f07e4-5f1a-4343-a047-4b0015ab2547 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.542276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 957/1908] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:35 2022] PUT /identity/v3/projects/dbf20b76ca594cfeb4076e4eae5cd1b3/users/eb14084591c84c348fedf9e003c4c49a/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:35.550278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.550278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.550278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.572518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:35.573950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.600985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87025e242699449690c069a278947f71'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.611036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:35.616660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87025e242699449690c069a278947f71'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.661809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 958/1909] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:35.671516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.672848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.703323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87025e242699449690c069a278947f71'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.712221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87025e242699449690c069a278947f71'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.712926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-50e988e2-2faf-4a5e-bf0c-06966292494e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87025e242699449690c069a278947f71'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:35.719152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 952/1910] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:35 2022] DELETE /identity/v3/users/87025e242699449690c069a278947f71 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:35.724185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.724496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.725126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.737305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e3eca-8f4b-4702-a904-39e2209fd9cc None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.746121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:35.747670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.794064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.795072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.795349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.795687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.797562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.810015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:35.821060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 953/1911] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:35.825881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.828177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.841103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8131c9-3abf-4b8b-8514-673116aeeb29 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.846901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.847171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.847446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.847721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] PATH_INFO: `/v3/projects/4cd695e199564c5d95872dd6f7f39bbe` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.849960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd695e199564c5d95872dd6f7f39bbe)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.852482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.888216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.889696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.900061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 959/1912] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:35.900807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:35.902204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:35.913049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cf5aa-6c53-44f1-8c59-0a1483ee493f service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:35.918803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:35.919511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:35.919795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:35.920075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:35.935963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:35.937569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117815eb-f238-4aba-9c58-636b7fde38a5 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:35.953263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 960/1913] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:35.990709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd695e199564c5d95872dd6f7f39bbe', 'request_id': 'req-5da39ad5-cd95-49eb-bc1f-19eac61bece1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.084371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd695e199564c5d95872dd6f7f39bbe', 'request_id': 'req-5da39ad5-cd95-49eb-bc1f-19eac61bece1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.093041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd695e199564c5d95872dd6f7f39bbe', 'request_id': 'req-5da39ad5-cd95-49eb-bc1f-19eac61bece1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.106104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] The token cache is being invalidate because project 4cd695e199564c5d95872dd6f7f39bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:36.106622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.107319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5da39ad5-cd95-49eb-bc1f-19eac61bece1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.109026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 954/1914] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:35 2022] DELETE /identity/v3/projects/4cd695e199564c5d95872dd6f7f39bbe => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:36.114726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.115206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.115504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.143760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:36.146005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:36.259124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:36.263034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 961/1915] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:36.271751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.272038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.272314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.300738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:36.302484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:36.311311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 955/1916] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:36.316337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.326885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235df9be-0ef2-4860-be65-68cf1fbdea56 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.333175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.334122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.334586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.335630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.336578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:36.337162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.339299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.352965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 962/1917] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:36.357321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.370392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fcb5e-71b1-4977-925f-c47dddabe9aa None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.377625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.378857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.379902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.380566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.381450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.383877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.482286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 956/1918] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:36.486524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.497238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a07154-2e85-4f5d-b4dc-9f2f49a6d680 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.520305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.522299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:36.568329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.569438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.569725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.570060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.570913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.573616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.608746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad20d1-fc86-4088-b84a-c0dae71f9f4c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.614126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.614815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.618394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.618394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] PATH_INFO: `/v3/projects/616e24958e89424a8bb4ab458c24b3af` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.618394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=616e24958e89424a8bb4ab458c24b3af)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.618806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.689398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 963/1919] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:36.693866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.708163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6182a-9119-456c-89ca-39f24b8984c9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.715015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.715968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.716264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.717214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.718002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.724246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.733770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 964/1920] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:36.737650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.765299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd25019-fa33-49f0-a3c9-fecb9245fff2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.774664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616e24958e89424a8bb4ab458c24b3af', 'request_id': 'req-d014765c-0409-4c6d-99f7-d4b094b51adc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.796511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:36.848668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.849611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.849877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.850148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.850849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.852564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.874411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 965/1921] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:36.875522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:36.888738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f243e-659c-437d-8d22-e48bd31173cb admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:36.893667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:36.894500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.894809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.895125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] PATH_INFO: `/v3/projects/987308f112bc4049a5f16d2b4ac11554/users/642aa54fa4b14c1c8c94efa08a92ae7c/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.914216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987308f112bc4049a5f16d2b4ac11554, user_id=642aa54fa4b14c1c8c94efa08a92ae7c, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:36.917117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652735d-1404-4f55-8bf4-9f2cc7696318 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:36.917799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616e24958e89424a8bb4ab458c24b3af', 'request_id': 'req-d014765c-0409-4c6d-99f7-d4b094b51adc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.919764 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:36.924937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616e24958e89424a8bb4ab458c24b3af', 'request_id': 'req-d014765c-0409-4c6d-99f7-d4b094b51adc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.938082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] The token cache is being invalidate because project 616e24958e89424a8bb4ab458c24b3af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:36.938452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.939294 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d014765c-0409-4c6d-99f7-d4b094b51adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:36.944567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 957/1922] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:36 2022] DELETE /identity/v3/projects/616e24958e89424a8bb4ab458c24b3af => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:36.946317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde89d4-e6a8-4870-a3eb-e78fc12b475b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.946694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde89d4-e6a8-4870-a3eb-e78fc12b475b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.947078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde89d4-e6a8-4870-a3eb-e78fc12b475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.967503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-afde89d4-e6a8-4870-a3eb-e78fc12b475b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:36.968916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-afde89d4-e6a8-4870-a3eb-e78fc12b475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:36.978659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 958/1923] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:36.982851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 966/1924] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:36 2022] PUT /identity/v3/projects/987308f112bc4049a5f16d2b4ac11554/users/642aa54fa4b14c1c8c94efa08a92ae7c/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:36.986605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.986951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.987286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:36.995777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:36.996126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:36.996697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.011811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:37.013158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:37.030712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] MFA Rules not processed for user `642aa54fa4b14c1c8c94efa08a92ae7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:37.031291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:37.085719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 959/1925] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:37.086624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.098520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fe38e-30a0-4bdd-8c08-f6054f14ede3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.105426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.105867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.106148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.106463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.107166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:37.107471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.109247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.112681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 967/1926] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:37.117638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.118929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:37.124123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 960/1927] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:37.132139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.132947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2916ce-87aa-4c6e-bccb-cf5f2f808831 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.137822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.138433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.138711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.138982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.139636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.141560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.142141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab5925-d6dc-4c38-97a7-95af9689ac4a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.147497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.148106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.148391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.148667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.149360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.151337 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.233627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 968/1928] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:37.237674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.263016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 961/1929] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:37.268893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.311176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ef8b9-bca2-46c7-b65f-d3a28c4da773 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.335025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de3e2-b033-4c7d-a43a-19ef189ae81c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.340191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.340828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.341139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.341576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.342259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.344835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.376534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.377195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.377474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.377744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.378412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.381461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.423980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 962/1930] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:37.425818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.436327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60c334-9972-4467-9075-c2f6effdb51e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.451006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.451662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.451909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.452185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.452894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.454523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.468674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 963/1931] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:37.474552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.485965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08e-d708-44f8-b82a-fb9bce6d1432 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.491164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.491792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.492077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.492388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.493440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.495399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.505823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 969/1932] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:37.508549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.515332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 964/1933] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:37.521805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.541200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be074871-10b0-4259-b7f0-4507306714de admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.545477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5d6b2-9821-41f9-94c6-348951aaf772 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.546562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.547378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.547765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.548164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] PATH_INFO: `/v3/projects/b8fd223d63d2438aa2e45ec9f9de6633/users/455e8556fbcc40fe8e4f01d620d50804/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.559319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.560204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.560635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.563166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.563166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.564063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.571851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fd223d63d2438aa2e45ec9f9de6633, user_id=455e8556fbcc40fe8e4f01d620d50804, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.575383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda3c57-7524-4b2f-82f9-de1c57de4a06 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.578457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 970/1934] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:37.585753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.605987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da13b-e02d-424c-9597-7237d91cc083 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.611536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.612167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.612877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.612877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] PATH_INFO: `/v3/projects/94fefbc48f4c4004924bbf774555f494/users/0a1af50a96904a49be05509d45ed4ac0/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.638430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fefbc48f4c4004924bbf774555f494, user_id=0a1af50a96904a49be05509d45ed4ac0, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.641162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.661930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 965/1935] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:37 2022] PUT /identity/v3/projects/b8fd223d63d2438aa2e45ec9f9de6633/users/455e8556fbcc40fe8e4f01d620d50804/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:37.672311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d44300-9068-4461-9003-2b1b919f611e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.672657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d44300-9068-4461-9003-2b1b919f611e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.672884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d44300-9068-4461-9003-2b1b919f611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.702828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-15d44300-9068-4461-9003-2b1b919f611e None None] MFA Rules not processed for user `455e8556fbcc40fe8e4f01d620d50804`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:37.704050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-15d44300-9068-4461-9003-2b1b919f611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:37.742367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 971/1936] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:37 2022] PUT /identity/v3/projects/94fefbc48f4c4004924bbf774555f494/users/0a1af50a96904a49be05509d45ed4ac0/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:37.749028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.764949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:37.773954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 966/1937] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:37.779361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d44300-9068-4461-9003-2b1b919f611e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.824084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a8eb5-f1fd-4c07-a21f-b595b160f5f1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.830719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.831548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.833162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.833162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.835573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.837432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.854886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 972/1938] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:37.861906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:37.898093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d44300-9068-4461-9003-2b1b919f611e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.900890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5af2a-93cf-42fa-a52b-6d6a49b6f77d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:37.903546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.904588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.904978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.905484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.906392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.908147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:37.908980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d46d3-9daa-4118-a58d-1ed76c129e4d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.909802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.910091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.910368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] PATH_INFO: `/v3/projects/94fefbc48f4c4004924bbf774555f494/users/0a1af50a96904a49be05509d45ed4ac0/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.915557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fefbc48f4c4004924bbf774555f494, user_id=0a1af50a96904a49be05509d45ed4ac0, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:37.918773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc0371-0e1a-4465-a0e6-bd5246157394 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:37.939755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 973/1939] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:37 2022] PUT /identity/v3/projects/94fefbc48f4c4004924bbf774555f494/users/0a1af50a96904a49be05509d45ed4ac0/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:37.952559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:37.952897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:37.960284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:37.979643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:37.994304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] MFA Rules not processed for user `0a1af50a96904a49be05509d45ed4ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:37.995872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:38.037634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 967/1940] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/projects => generated 377 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:38.038902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.039248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.039532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.076230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] MFA Rules not processed for user `eb14084591c84c348fedf9e003c4c49a`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:38.078554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:38.081852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 974/1941] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:38.082986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.174376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 968/1942] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:38.174376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.175747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:38.183298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec4e3f-9830-45ba-9087-43a009494e52 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.263052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a0b9b-3817-47c7-822a-5a470db748f2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.269149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.269793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.271117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.271405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.280529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.282344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.282808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.283327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.284161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.286996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.350909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.352355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.364388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 969/1943] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:38.364388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.365597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:38.379984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec8cf-6f0f-418f-bfc8-de43eed54f35 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.403314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.403963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.404249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.404355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 975/1944] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:38 2022] POST /identity/v3/users => generated 289 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:38.404685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.405426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.408395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.408543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.445141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360e956-696d-41d0-86a8-d816acea77b9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.457099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.458395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.458679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.458970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.461945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.465030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.488753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 976/1945] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:38.490693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.533155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6387-0a7d-4cc2-825a-0360ba98a94a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.535659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 970/1946] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:38 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:38.536829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.610315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb8d46-8656-46f5-be77-6c6957606c84 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.618864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.620235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.620521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.620799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.624742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.627959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.628499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.628957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.630377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.633277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.649285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 977/1947] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:38.654155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.664432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5234-c65d-4e36-a19d-5772e610b6ed admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.670065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.670792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.671149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.671481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.672219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.674230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.704490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.706147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.712335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:38.718032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 971/1948] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:38.718965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.731651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb69f37-56af-4f68-ae99-458d49215e1e service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.736932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.737814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.738249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.738555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] PATH_INFO: `/v3/projects/fe4d8bb0dfaf4b7691d0f3ea909dffa3/users/90066723044f4fd9b7eb428adb24da98/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.760883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4d8bb0dfaf4b7691d0f3ea909dffa3, user_id=90066723044f4fd9b7eb428adb24da98, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.761697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b200b-b5a6-494b-8c7c-c4366f7f87e6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.784310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 978/1949] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:38 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:38.794855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.847443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c50f-8f56-46a6-8c15-8374416c4b3f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:38.861799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:38.862449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.862728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.863003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.865022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:38.869281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:38.876597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 972/1950] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:38 2022] PUT /identity/v3/projects/fe4d8bb0dfaf4b7691d0f3ea909dffa3/users/90066723044f4fd9b7eb428adb24da98/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:38.894645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:38.895545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:38.895979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:38.917781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 979/1951] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:38.923461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:38.937682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] MFA Rules not processed for user `90066723044f4fd9b7eb428adb24da98`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:38.939836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:38.962863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e10c0-6678-41ed-a31d-ae91934660b4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.056111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 973/1952] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:39.061275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.101289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.101918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.102197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.102476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] PATH_INFO: `/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf/users/36ddd826cfee47dd9e1185392c3d8e36/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.147254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44c8b6ad82f43b18344039ee2db5ecf, user_id=36ddd826cfee47dd9e1185392c3d8e36, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.168310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.193686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f96cbf-d325-40f9-8e50-a034397cfb0a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.200893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.201905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.202313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.203040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.204735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.208978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.266895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 980/1953] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:38 2022] PUT /identity/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf/users/36ddd826cfee47dd9e1185392c3d8e36/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:39.268039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.351380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5dd72-ac69-4e47-919b-dc1d560b8526 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.357986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.359012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.371430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.372653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.374894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.375038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 974/1954] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:39 2022] POST /identity/v3/projects => generated 377 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:39.378613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.380642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.406036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 981/1955] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:39.411271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.423094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f296917-cb8e-493d-95f0-2f22caff922a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.477897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469afbea-4392-431b-aa8e-bc1d6cd153d5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.483379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.484009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.484292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.484589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.485280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.487216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.494854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.495620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.496027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.496304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] PATH_INFO: `/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf/users/36ddd826cfee47dd9e1185392c3d8e36/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.498586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44c8b6ad82f43b18344039ee2db5ecf, user_id=36ddd826cfee47dd9e1185392c3d8e36, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.501619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.513247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 982/1956] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:39 2022] PUT /identity/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf/users/36ddd826cfee47dd9e1185392c3d8e36/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:39.581545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 975/1957] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:39 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:39.585770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.637053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:39.658043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce478fc5-a724-4bf4-9a7b-e0a5b32048a9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.663464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.664197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.664529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.664849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.665782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.667546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.682008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 983/1958] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:39.687125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.695068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07874ad-6a91-4464-9d10-6152f747bc8a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.700622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.701386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.701664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.701942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] PATH_INFO: `/v3/projects/3cff3d024cb24ca19ae2395581d820e9/users/0bfa94cefeb8478380f44f2526535a76/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.722229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cff3d024cb24ca19ae2395581d820e9, user_id=0bfa94cefeb8478380f44f2526535a76, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.725180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.813082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 976/1959] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:39 2022] PUT /identity/v3/projects/3cff3d024cb24ca19ae2395581d820e9/users/0bfa94cefeb8478380f44f2526535a76/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:39.817828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.827896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476961e-0b27-4e86-be47-8a768df76f7c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.846660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:39.891402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.892406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.892697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.892974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.894305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.895977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ed9c1-3b71-4d4b-8f6d-9b58e4a32176 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.911792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 984/1960] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:39.913089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:39.924430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f9a2-2e44-4ecc-9bfb-43dfa7301275 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:39.932260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:39.933702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.933987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.934264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] PATH_INFO: `/v3/projects/3cff3d024cb24ca19ae2395581d820e9/users/0bfa94cefeb8478380f44f2526535a76/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.936276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cff3d024cb24ca19ae2395581d820e9, user_id=0bfa94cefeb8478380f44f2526535a76, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:39.939477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:39.951637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 977/1961] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:39 2022] PUT /identity/v3/projects/3cff3d024cb24ca19ae2395581d820e9/users/0bfa94cefeb8478380f44f2526535a76/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:39.962994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201521aa-b000-401f-ae4b-3ac2f37692ed None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:39.962994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201521aa-b000-401f-ae4b-3ac2f37692ed None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:39.962994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201521aa-b000-401f-ae4b-3ac2f37692ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:39.996714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-201521aa-b000-401f-ae4b-3ac2f37692ed None None] MFA Rules not processed for user `0bfa94cefeb8478380f44f2526535a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:39.998235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-201521aa-b000-401f-ae4b-3ac2f37692ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:40.084725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 985/1962] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:40.097132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:40.166556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdbc82-fdc8-4720-8fd4-a8c8b52664fa admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:40.170591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:40.171329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:40.171741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:40.172119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:40.260761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:40.263113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:40.289707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 978/1963] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:40.587780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:41.001368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:41.364811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:41.365479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:41.365953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:41.399886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] MFA Rules not processed for user `36ddd826cfee47dd9e1185392c3d8e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:41.436767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:41.498555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 986/1964] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:41 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:41.507406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:41.508315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:41.523517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748b687-49b7-48b5-9d8c-d7dd7644c40f service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:41.527234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:41.527935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:41.528283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:41.528889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:41.592064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:41.593636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e3d4-3a4a-4764-958f-d4180214ab8f service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:41.604139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 979/1965] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:41.897647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:42.250199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:42.267730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:42.339125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c72da-79d3-41d0-9e68-2b97927c052a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:42.345975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:42.346669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:42.347015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:42.347464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] PATH_INFO: `/v3/users/642aa54fa4b14c1c8c94efa08a92ae7c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:42.349428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642aa54fa4b14c1c8c94efa08a92ae7c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:42.351682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:42.529428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642aa54fa4b14c1c8c94efa08a92ae7c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:42.541685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642aa54fa4b14c1c8c94efa08a92ae7c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:42.548481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642aa54fa4b14c1c8c94efa08a92ae7c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:42.549192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642aa54fa4b14c1c8c94efa08a92ae7c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:42.645502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642aa54fa4b14c1c8c94efa08a92ae7c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:42.653376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 987/1966] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:42 2022] DELETE /identity/v3/users/642aa54fa4b14c1c8c94efa08a92ae7c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:42.658437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:42.659923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:42.660433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:42.681291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:42.682525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:42.751738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 980/1967] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:42.764360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:42.765885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:42.826875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:42.834726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc32c-8a7b-4cd2-88c3-cd970f7c83c1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:42.839889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:42.840780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:42.841282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:42.841561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:42.854477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:42.857972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:42.869636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 988/1968] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:43.364289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:43.364289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:43.381757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7834f38-182d-4f2b-8095-76b2911dd07a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:43.393938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:43.395632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:43.395947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:43.396227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] PATH_INFO: `/v3/projects/987308f112bc4049a5f16d2b4ac11554` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:43.397454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=987308f112bc4049a5f16d2b4ac11554)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:43.399401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:43.515030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987308f112bc4049a5f16d2b4ac11554', 'request_id': 'req-6cfa91b3-e050-402b-8b32-cdf7d768a71b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.515409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:43.516935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:43.524067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:43.586482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ab945-2798-44ca-8fe0-20726d8d533e service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:43.592152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987308f112bc4049a5f16d2b4ac11554', 'request_id': 'req-6cfa91b3-e050-402b-8b32-cdf7d768a71b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.600874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987308f112bc4049a5f16d2b4ac11554', 'request_id': 'req-6cfa91b3-e050-402b-8b32-cdf7d768a71b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.613306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] The token cache is being invalidate because project 987308f112bc4049a5f16d2b4ac11554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:43.614028 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.615220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.617542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 981/1969] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:43 2022] DELETE /identity/v3/projects/987308f112bc4049a5f16d2b4ac11554 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:43.621158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:43.622729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:43.666818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:43.667462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:43.667723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:43.667991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:43.670386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:43.683703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:43.703823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa91b3-e050-402b-8b32-cdf7d768a71b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:43.719327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:43.721716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:43.722502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:43.723764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] PATH_INFO: `/v3/users/0a1af50a96904a49be05509d45ed4ac0` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:43.726286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1af50a96904a49be05509d45ed4ac0)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:43.728224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:43.746328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:43.748452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:43.759650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 989/1970] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:43.929054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1af50a96904a49be05509d45ed4ac0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:43.941430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1af50a96904a49be05509d45ed4ac0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.014986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1af50a96904a49be05509d45ed4ac0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.023563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1af50a96904a49be05509d45ed4ac0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.024256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1af50a96904a49be05509d45ed4ac0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.030955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 982/1971] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:43 2022] DELETE /identity/v3/users/0a1af50a96904a49be05509d45ed4ac0 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:44.519912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:44.521406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:44.531885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33bb11-1f7a-4b70-ba50-f44cb3f31dca service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:44.580898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:44.581413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:44.581715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:44.581996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:44.584199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:44.589290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:44.592026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:44.624108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:44.663819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:44.665292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f6adc-c593-4a77-8cae-4a02e130bfc0 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:44.665405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1137a-9fcb-4c3c-9460-cb93d93e698f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:44.670447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:44.671091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:44.671387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:44.671667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] PATH_INFO: `/v3/projects/94fefbc48f4c4004924bbf774555f494` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:44.672891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fefbc48f4c4004924bbf774555f494)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:44.674901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:44.675227 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 990/1972] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:44.839408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fefbc48f4c4004924bbf774555f494', 'request_id': 'req-b353988f-0bfb-45e8-83c9-256f787067d9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.906418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fefbc48f4c4004924bbf774555f494', 'request_id': 'req-b353988f-0bfb-45e8-83c9-256f787067d9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.917042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fefbc48f4c4004924bbf774555f494', 'request_id': 'req-b353988f-0bfb-45e8-83c9-256f787067d9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.932227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:44.937890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] The token cache is being invalidate because project 94fefbc48f4c4004924bbf774555f494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:44.938430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.939512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b353988f-0bfb-45e8-83c9-256f787067d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:44.945145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 983/1973] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:44 2022] DELETE /identity/v3/projects/94fefbc48f4c4004924bbf774555f494 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:44.970326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:44.970326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:44.970326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:44.987548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:44.988836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:45.137471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 991/1974] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:45.145052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.145408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.146416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.179149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:45.180653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:45.191802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 984/1975] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:45.196485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.207681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f2f9-d1ba-4ac1-a6c7-8cf7de6950c7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.213379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.214066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.214332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.214607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.215319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:45.215626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.217291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.233169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 992/1976] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:36:45.235233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.249333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cbeff-7a3f-4ab8-842b-a91839ca37c6 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.256416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.257110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.258014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.258014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.260192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.262925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.404833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 985/1977] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:45 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:45.405903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.424406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a063816-7b19-4fb5-aac0-bfdcc14c9447 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.528246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.530567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:45.542788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.543928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.544206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.544427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.545129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.547056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.599448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89d9e5-47f3-4ba8-9795-c2c4a064cd9a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.603290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.604021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.604382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.604724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.606795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:45.647040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 993/1978] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:45 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:45.651542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.672910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f6c22-f671-4b8d-8e22-43b7a04d09e9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.680370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.680914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.681226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.681765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.681896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.682312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.683611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.686295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.701208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 994/1979] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:45.705965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.710204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 986/1980] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:45.719282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e08c2-64c3-4450-950a-75ce7d45fe63 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.735420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.736253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.737137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.737808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.738889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.742247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a21e3b-7d12-4a6f-a248-631b741f7469 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.758181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 995/1981] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:45.762410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:45.774613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a41f-986c-4473-96c5-f47982dfc5f8 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:45.779552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:45.780252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.780538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.780822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] PATH_INFO: `/v3/projects/f2cf6a56a3a94de5ae8997751f386271/users/8299b2d7b43a45509524f117fc445c17/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.799705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2cf6a56a3a94de5ae8997751f386271, user_id=8299b2d7b43a45509524f117fc445c17, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:45.803321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:45.913621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 987/1982] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:45 2022] PUT /identity/v3/projects/f2cf6a56a3a94de5ae8997751f386271/users/8299b2d7b43a45509524f117fc445c17/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:45.924178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:45.924463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:45.924787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:45.966616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] MFA Rules not processed for user `8299b2d7b43a45509524f117fc445c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:45.967994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:45.974287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:46.060585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 996/1983] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:46.062423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:46.167686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:46.174291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ed831-feb0-4721-99db-6eb235201438 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:46.182236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:46.183793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:46.184088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:46.184367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:46.185096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:46.187020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:46.355651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 988/1984] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:46 2022] POST /identity/v3/projects => generated 389 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:46.361218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:46.375745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431364aa-714a-44fe-826f-2e2f19c09af4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:46.449726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:46.451437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:46.452074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:46.452074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:46.453076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:46.455241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:46.585483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 997/1985] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:46 2022] POST /identity/v3/users => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:46.589721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:46.600786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbfd5e-aac2-46cb-9d3b-c3d41293803c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:46.606075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:46.612258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:46.612258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:46.612258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:46.612258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:46.612258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:46.626263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 989/1986] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:46.627405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:46.645396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210478-1bad-44a6-82f8-107314bfbc6b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:46.654295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:46.655298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:46.655748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:46.656174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] PATH_INFO: `/v3/projects/a188bc77f5884c079b8040cfe28d13c2/users/27222507051146128a09f83b37688a6d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:46.676654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188bc77f5884c079b8040cfe28d13c2, user_id=27222507051146128a09f83b37688a6d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:46.679974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:46.775890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 998/1987] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:46 2022] PUT /identity/v3/projects/a188bc77f5884c079b8040cfe28d13c2/users/27222507051146128a09f83b37688a6d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:46.780135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:46.791767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36773d54-a169-432c-8c1d-299a664431dc admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:46.971813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:46.972582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:46.972825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:46.973142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:46.974088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:46.975774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc767e42-f3a6-4288-8e99-186f53d3675b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:47.025086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 990/1988] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:47.032507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:47.046798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213bd29-ff36-4f5f-88bf-fef5212c1610 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:47.060928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:47.085159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:47.085608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:47.085969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] PATH_INFO: `/v3/projects/a188bc77f5884c079b8040cfe28d13c2/users/27222507051146128a09f83b37688a6d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:47.097899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188bc77f5884c079b8040cfe28d13c2, user_id=27222507051146128a09f83b37688a6d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:47.102132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:47.161108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 999/1989] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:47 2022] PUT /identity/v3/projects/a188bc77f5884c079b8040cfe28d13c2/users/27222507051146128a09f83b37688a6d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:47.164836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:47.165280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:47.165700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:47.220709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] MFA Rules not processed for user `27222507051146128a09f83b37688a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:47.225031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:47.372320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 991/1990] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:47.628434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:47.631177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:47.714053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b710a-3133-42a2-8504-1057ff752b14 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:47.732880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:47.733590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:47.733929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:47.734210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:47.758695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:47.761747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e8581-fd66-448e-9d82-e39f0bfe42bd service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:47.785938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1000/1991] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:48.038836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:48.113843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:48.128494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff71bd-63e7-4c74-bff8-b75d255abcb8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:48.134495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:48.135216 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:48.135622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:48.136095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] PATH_INFO: `/v3/users/8299b2d7b43a45509524f117fc445c17` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:48.138856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8299b2d7b43a45509524f117fc445c17)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:48.143745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:48.391432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8299b2d7b43a45509524f117fc445c17'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:48.406730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8299b2d7b43a45509524f117fc445c17'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:48.488031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8299b2d7b43a45509524f117fc445c17'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:48.500119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8299b2d7b43a45509524f117fc445c17'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:48.501434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8299b2d7b43a45509524f117fc445c17'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:48.513561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 992/1992] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:48 2022] DELETE /identity/v3/users/8299b2d7b43a45509524f117fc445c17 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:48.518185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:48.518694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:48.519125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:48.530031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:48.571247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:48.576374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:48.676445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1001/1993] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:48.687374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:48.688798 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:48.766261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4481a47-75f3-4b2f-b9b8-4e015e58e511 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:48.770273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:48.771451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:48.771998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:48.772341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:48.782706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:48.784167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:48.796110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 993/1994] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:49.314913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:49.316412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:49.359861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00765-feaf-4d7d-bc73-74c23eb4caa9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:49.372623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:49.373628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:49.374325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:49.374817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] PATH_INFO: `/v3/projects/f2cf6a56a3a94de5ae8997751f386271` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:49.379844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2cf6a56a3a94de5ae8997751f386271)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:49.382682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:49.527412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2cf6a56a3a94de5ae8997751f386271', 'request_id': 'req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.533456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2cf6a56a3a94de5ae8997751f386271', 'request_id': 'req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.649386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2cf6a56a3a94de5ae8997751f386271', 'request_id': 'req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.663696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] The token cache is being invalidate because project f2cf6a56a3a94de5ae8997751f386271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:49.663848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.664551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.666341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1002/1995] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:49 2022] DELETE /identity/v3/projects/f2cf6a56a3a94de5ae8997751f386271 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:49.670780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:49.682726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87b6e6-35fe-4027-9612-c1d42579e0b7 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:49.739124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:49.740502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:49.740815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:49.741136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] PATH_INFO: `/v3/users/27222507051146128a09f83b37688a6d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:49.742352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] RBAC: Authorizing `identity:delete_user(user_id=27222507051146128a09f83b37688a6d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:49.744279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:49.759605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:49.974389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27222507051146128a09f83b37688a6d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:49.986462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27222507051146128a09f83b37688a6d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:50.073717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27222507051146128a09f83b37688a6d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:50.079169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27222507051146128a09f83b37688a6d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:50.079964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b59050ac-c82f-438b-824f-be788e04677b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27222507051146128a09f83b37688a6d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:50.087055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 994/1996] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:49 2022] DELETE /identity/v3/users/27222507051146128a09f83b37688a6d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:50.796571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:50.798050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:50.812891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbad5f-7cfe-4ad4-9e5f-a822bc1af08f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:50.871812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:50.872460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:50.872741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:50.873037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] PATH_INFO: `/v3/projects/a188bc77f5884c079b8040cfe28d13c2` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:50.874385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a188bc77f5884c079b8040cfe28d13c2)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:50.876289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:50.990685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:51.028567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a188bc77f5884c079b8040cfe28d13c2', 'request_id': 'req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.035704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a188bc77f5884c079b8040cfe28d13c2', 'request_id': 'req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.106036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a188bc77f5884c079b8040cfe28d13c2', 'request_id': 'req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.139690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] The token cache is being invalidate because project a188bc77f5884c079b8040cfe28d13c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:51.140761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.144647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2b6ce873-a94c-4c7a-af2d-63f521e6be0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.146640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1003/1997] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:50 2022] DELETE /identity/v3/projects/a188bc77f5884c079b8040cfe28d13c2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:51.152664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.153166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.153243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.195535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:51.198907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:51.279214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 995/1998] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:51.287185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.288026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.288314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.311537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:51.312749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:51.322487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1004/1999] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:51.327036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.338861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46351fed-0516-452b-8fd3-c025a77fdac0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.343552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.344178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.344459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.344732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.345477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:51.346404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.347416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.360195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 996/2000] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:36:51.364720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.374709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a692cd-5afb-444f-a1f6-5420039635d2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.380676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.382779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.382779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.382779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.383152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.384945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.485034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1005/2001] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:51.487576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.498917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd184a-85ba-4de5-b6f6-3210a817a25f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.570960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.571613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.571893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.572156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.572813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.574867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.653150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 997/2002] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:36:51.657569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.668813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361baaa-8841-4f7f-b723-3d0939e5ede0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.674460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.674460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.674638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.674837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.675663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.677806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.691437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1006/2003] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:51.692105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.692491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.694793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:51.705295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2d3a9-d629-4901-971c-de672982fee7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.706846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22be3d-ffee-4b68-aa73-fbeeff8e20a9 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.711023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.711671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.711952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.712192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.712303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.712836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.712947 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.713153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.713431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] PATH_INFO: `/v3/users/455e8556fbcc40fe8e4f01d620d50804` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.714557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.714673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455e8556fbcc40fe8e4f01d620d50804)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.716546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.721588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:51.726147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1007/2004] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:51.733403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.740378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666968b1-4375-42d3-a63a-461cc7e7f681 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:51.745179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:51.745846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.746130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.746423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] PATH_INFO: `/v3/projects/a8ee55d2489d40f480f2fc2772712744/users/227203ab050f43f793d8c4fe3c31497d/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.761250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ee55d2489d40f480f2fc2772712744, user_id=227203ab050f43f793d8c4fe3c31497d, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:51.763896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a15fcd-cacd-401c-879d-ad347559278f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:51.826508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1008/2005] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:51 2022] PUT /identity/v3/projects/a8ee55d2489d40f480f2fc2772712744/users/227203ab050f43f793d8c4fe3c31497d/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:51.831552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.831786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.832089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:51.850972 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:51.852181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:51.883186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455e8556fbcc40fe8e4f01d620d50804'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.895792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455e8556fbcc40fe8e4f01d620d50804'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.914241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1009/2006] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:51.930252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:51.932171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:51.966808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455e8556fbcc40fe8e4f01d620d50804'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.980300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455e8556fbcc40fe8e4f01d620d50804'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.980680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-321bc857-458d-412e-871d-82aeb7003a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455e8556fbcc40fe8e4f01d620d50804'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:51.992057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 998/2007] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:51 2022] DELETE /identity/v3/users/455e8556fbcc40fe8e4f01d620d50804 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:51.996557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:51.996920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:51.997237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:52.018809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:52.020167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:52.024233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:52.045956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84018b32-7a16-4609-a5e2-aa4ede841432 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:52.110456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 999/2008] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:52.129852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:52.131450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:52.131799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:52.132370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:52.134938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:52.135105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:52.137192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:52.153376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:52.155393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:52.170041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1010/2009] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:52.175442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710feee-2d8a-4392-87c3-8c74a41f0e01 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:52.182700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:52.183628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:52.184094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:52.184570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:52.202208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:52.202977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:52.229093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1000/2010] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:36:53.371175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:53.374627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:53.396940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44222c45-443e-4fc2-acf6-c91ba4bb478f service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:53.403778 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:53.404763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:53.405371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:53.405845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] PATH_INFO: `/v3/projects/b8fd223d63d2438aa2e45ec9f9de6633` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:53.407902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fd223d63d2438aa2e45ec9f9de6633)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:53.410455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:53.568088 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fd223d63d2438aa2e45ec9f9de6633', 'request_id': 'req-abe75fbf-d480-446d-a20f-80fc3f80b07d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.575049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fd223d63d2438aa2e45ec9f9de6633', 'request_id': 'req-abe75fbf-d480-446d-a20f-80fc3f80b07d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.660626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fd223d63d2438aa2e45ec9f9de6633', 'request_id': 'req-abe75fbf-d480-446d-a20f-80fc3f80b07d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.681633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] The token cache is being invalidate because project b8fd223d63d2438aa2e45ec9f9de6633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:53.681837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.682572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-abe75fbf-d480-446d-a20f-80fc3f80b07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.684894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1011/2011] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:53 2022] DELETE /identity/v3/projects/b8fd223d63d2438aa2e45ec9f9de6633 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:53.688967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:53.702347 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7a35-ad2b-4c3e-9123-813743ba561f service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:53.773236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:53.774047 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:53.774616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:53.774980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] PATH_INFO: `/v3/users/90066723044f4fd9b7eb428adb24da98` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:53.776672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=90066723044f4fd9b7eb428adb24da98)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:53.778951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:53.792610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:53.983525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90066723044f4fd9b7eb428adb24da98'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:53.996523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90066723044f4fd9b7eb428adb24da98'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:54.115939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90066723044f4fd9b7eb428adb24da98'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:54.124289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90066723044f4fd9b7eb428adb24da98'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:54.124913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90066723044f4fd9b7eb428adb24da98'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:54.131625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1001/2012] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:53 2022] DELETE /identity/v3/users/90066723044f4fd9b7eb428adb24da98 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:54.534688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:54.537567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:54.537567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:54.584373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] MFA Rules not processed for user `227203ab050f43f793d8c4fe3c31497d`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:54.587252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:54.670359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1012/2013] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:54.682258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:54.684109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:54.751077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df4c3-7a0f-4877-a52e-a701554593ba admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:54.761686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:54.762323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:54.762624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:54.762881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:54.775076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:54.777147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:54.790809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1002/2014] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:54.907886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:54.910131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:54.931016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:55.007750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf48324-974f-4c19-87bd-8ef55fd5d85a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:55.016393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:55.017171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:55.021572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:55.021572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] PATH_INFO: `/v3/projects/fe4d8bb0dfaf4b7691d0f3ea909dffa3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:55.021572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4d8bb0dfaf4b7691d0f3ea909dffa3)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:55.023442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:55.205321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4d8bb0dfaf4b7691d0f3ea909dffa3', 'request_id': 'req-5e43ff27-a08f-4ff8-8861-eab285dfd607'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.214900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4d8bb0dfaf4b7691d0f3ea909dffa3', 'request_id': 'req-5e43ff27-a08f-4ff8-8861-eab285dfd607'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.287536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4d8bb0dfaf4b7691d0f3ea909dffa3', 'request_id': 'req-5e43ff27-a08f-4ff8-8861-eab285dfd607'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.303347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] The token cache is being invalidate because project fe4d8bb0dfaf4b7691d0f3ea909dffa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:55.303347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.303347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.304194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1013/2015] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:54 2022] DELETE /identity/v3/projects/fe4d8bb0dfaf4b7691d0f3ea909dffa3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:55.308747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:55.325950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6a805-b9f3-4f20-a003-3ff2c764b3a1 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:55.391668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:55.392365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:55.392654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:55.392932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] PATH_INFO: `/v3/users/0bfa94cefeb8478380f44f2526535a76` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:55.394205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfa94cefeb8478380f44f2526535a76)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:55.396706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:55.571433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:55.581094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfa94cefeb8478380f44f2526535a76'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.594148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfa94cefeb8478380f44f2526535a76'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.683758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfa94cefeb8478380f44f2526535a76'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.691265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfa94cefeb8478380f44f2526535a76'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.691910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f24cc710-2cba-4417-8b16-b56d43c791f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfa94cefeb8478380f44f2526535a76'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:55.698624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1003/2016] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:36:55 2022] DELETE /identity/v3/users/0bfa94cefeb8478380f44f2526535a76 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:36:56.102807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.104828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:56.120349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ff27-a08f-4ff8-8861-eab285dfd607 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.180053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.180698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.181011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.181285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] PATH_INFO: `/v3/projects/3cff3d024cb24ca19ae2395581d820e9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.182440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cff3d024cb24ca19ae2395581d820e9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.184601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.312237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:56.315182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cff3d024cb24ca19ae2395581d820e9', 'request_id': 'req-a59c53b2-deb9-4b42-8bf0-003f23241a49'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:56.322319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cff3d024cb24ca19ae2395581d820e9', 'request_id': 'req-a59c53b2-deb9-4b42-8bf0-003f23241a49'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:56.400861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cff3d024cb24ca19ae2395581d820e9', 'request_id': 'req-a59c53b2-deb9-4b42-8bf0-003f23241a49'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:56.412268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] The token cache is being invalidate because project 3cff3d024cb24ca19ae2395581d820e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:36:56.412618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:56.413293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a59c53b2-deb9-4b42-8bf0-003f23241a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:36:56.414845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1014/2017] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:36:56 2022] DELETE /identity/v3/projects/3cff3d024cb24ca19ae2395581d820e9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:56.419671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.420184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.420288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.440946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:56.442151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:56.502058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1004/2018] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:56.510012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.510310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.510588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.530217 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:56.531429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:56.539931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1015/2019] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:36:56.544113 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.554515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb717-f039-41a8-9ac9-82ef1849f385 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.559534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.560159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.560444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.560722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.561460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:36:56.561770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.563375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.575455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1005/2020] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:36:56.588247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.589909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1d5f4-a148-4679-8866-2530ef39b783 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.594826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.595452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.595735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.596046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.596716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.598666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.699461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1016/2021] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:36:56 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:56.703743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.714221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4c2b9-9d5f-4d39-aec2-6b8c2bc45fd2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.772221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.772832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.773171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.773490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.774149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.776085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.860773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1006/2022] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:36:56 2022] POST /identity/v3/users => generated 282 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:36:56.865756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.876128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8dea3-7e2b-4d0b-b8ea-15ded24212ed admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.881997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.882678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.883024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.883387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.884095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.885730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.896616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1017/2023] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:36:56.901174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.911859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24387c-6030-4507-a8d9-7b6f2e24896d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.916747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.918385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.918385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.918385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.918689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.920240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376f49-f5a7-452d-97c5-1176e99b56e7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:56.931669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1007/2024] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:36:56.936499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:56.948242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40a89a-fcc4-498b-a81c-13f9cf6dd867 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:56.953142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:56.953838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:56.954168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:56.954441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] PATH_INFO: `/v3/projects/6cefe8f925be47f0b74012553230aa46/users/a1fd257882f5457d8407383a926871f1/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:56.971311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cefe8f925be47f0b74012553230aa46, user_id=a1fd257882f5457d8407383a926871f1, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:56.974130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:57.076442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1018/2025] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:36:56 2022] PUT /identity/v3/projects/6cefe8f925be47f0b74012553230aa46/users/a1fd257882f5457d8407383a926871f1/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:36:57.088556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142d1b1-7ce6-41d5-afab-bf68f8691f01 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:57.088879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142d1b1-7ce6-41d5-afab-bf68f8691f01 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:57.089163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142d1b1-7ce6-41d5-afab-bf68f8691f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:57.119388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0142d1b1-7ce6-41d5-afab-bf68f8691f01 None None] MFA Rules not processed for user `a1fd257882f5457d8407383a926871f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:57.120540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0142d1b1-7ce6-41d5-afab-bf68f8691f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:57.125913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:36:57.195686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1008/2026] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:36:57.208484 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:36:57.210827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:57.275647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed6b2e-cb84-49ce-9c05-e688d5c83a4e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:36:57.279877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:36:57.280497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:57.280796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:57.281250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:57.292270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:36:57.293755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065eb7e-8ae9-4e11-8e11-fae27f823f07 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:36:57.304220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1019/2027] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:36:57 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:36:57.746893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156826-0f46-4cff-86ef-976f80d3bed2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:36:57.747440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156826-0f46-4cff-86ef-976f80d3bed2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:36:57.747683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156826-0f46-4cff-86ef-976f80d3bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:36:57.769622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6f156826-0f46-4cff-86ef-976f80d3bed2 None None] MFA Rules not processed for user `227203ab050f43f793d8c4fe3c31497d`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:36:57.771616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6f156826-0f46-4cff-86ef-976f80d3bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:36:57.833353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1009/2028] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:36:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:06.529667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:06.530136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:06.530136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:06.531728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1020/2029] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:37:06.568494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:06.569847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:06.570358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:06.601716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] MFA Rules not processed for user `60aee4f7a30845f69db2e71f02fa139b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:06.604902 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:06.703736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1010/2030] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:37:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:06.736466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:06.739279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:06.762699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e644ed-9149-4059-881f-232a78d1d241 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:06.769855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5bb-7371-42df-981c-496462906866 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:06.771948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5bb-7371-42df-981c-496462906866 service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:06.772577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5bb-7371-42df-981c-496462906866 service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:06.772920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5bb-7371-42df-981c-496462906866 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:06.783461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5bb-7371-42df-981c-496462906866 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:06.785171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5bb-7371-42df-981c-496462906866 service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:06.796068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1021/2031] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:07.420945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:07.422848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:07.476530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:07.492094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22990f-33ac-40bf-ba51-2181aba39a94 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:07.496728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:07.497478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:07.497760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:07.498033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] PATH_INFO: `/v3/users/eb14084591c84c348fedf9e003c4c49a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:07.499244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb14084591c84c348fedf9e003c4c49a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:07.501204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:07.779669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb14084591c84c348fedf9e003c4c49a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:07.796190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb14084591c84c348fedf9e003c4c49a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:07.895559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb14084591c84c348fedf9e003c4c49a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:07.909985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb14084591c84c348fedf9e003c4c49a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:07.915925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb14084591c84c348fedf9e003c4c49a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:07.928149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1011/2032] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:07 2022] DELETE /identity/v3/users/eb14084591c84c348fedf9e003c4c49a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:08.479763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5bb-7371-42df-981c-496462906866 service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:08.492121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5bb-7371-42df-981c-496462906866 service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:08.568846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:08.569632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:08.569923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:08.570183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] PATH_INFO: `/v3/projects/dbf20b76ca594cfeb4076e4eae5cd1b3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:08.571350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf20b76ca594cfeb4076e4eae5cd1b3)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:08.573337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:08.574993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:08.666689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf20b76ca594cfeb4076e4eae5cd1b3', 'request_id': 'req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:08.673321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf20b76ca594cfeb4076e4eae5cd1b3', 'request_id': 'req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:08.787053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf20b76ca594cfeb4076e4eae5cd1b3', 'request_id': 'req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:08.802702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] The token cache is being invalidate because project dbf20b76ca594cfeb4076e4eae5cd1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:08.807133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:08.807133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:08.807133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1022/2033] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:08 2022] DELETE /identity/v3/projects/dbf20b76ca594cfeb4076e4eae5cd1b3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:08.810483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:08.812059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:08.826033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72508fb2-0022-4c9a-9380-5d581ebaf2de admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:08.889069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:08.889858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:08.890148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:08.890426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] PATH_INFO: `/v3/users/36ddd826cfee47dd9e1185392c3d8e36` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:08.898092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ddd826cfee47dd9e1185392c3d8e36)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:08.900192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:09.135821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ddd826cfee47dd9e1185392c3d8e36'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.153511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ddd826cfee47dd9e1185392c3d8e36'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.160689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:09.266621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ddd826cfee47dd9e1185392c3d8e36'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.274666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ddd826cfee47dd9e1185392c3d8e36'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.279048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9315cd72-a487-481a-9267-ef16427a1ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ddd826cfee47dd9e1185392c3d8e36'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.281362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1012/2034] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:08 2022] DELETE /identity/v3/users/36ddd826cfee47dd9e1185392c3d8e36 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:09.557262 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:09.560168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:09.572632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c8db6-9cb9-4815-b45a-2b637ce8f6d2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:09.637102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:09.637801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:09.638079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:09.638351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] PATH_INFO: `/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:09.639494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44c8b6ad82f43b18344039ee2db5ecf)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:09.641460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:09.783677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44c8b6ad82f43b18344039ee2db5ecf', 'request_id': 'req-a4ad52c7-a633-4bbe-b069-8c048f81d69e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.790343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44c8b6ad82f43b18344039ee2db5ecf', 'request_id': 'req-a4ad52c7-a633-4bbe-b069-8c048f81d69e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.900558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44c8b6ad82f43b18344039ee2db5ecf', 'request_id': 'req-a4ad52c7-a633-4bbe-b069-8c048f81d69e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.912389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] The token cache is being invalidate because project b44c8b6ad82f43b18344039ee2db5ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:09.912741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.913378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a4ad52c7-a633-4bbe-b069-8c048f81d69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:09.915381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1023/2035] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:09 2022] DELETE /identity/v3/projects/b44c8b6ad82f43b18344039ee2db5ecf => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:09.920346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:09.932571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:09.932571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:09.941492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:09.942701 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:10.001671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1013/2036] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:10.010013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.012878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.012878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.030729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:10.035822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:10.041442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1024/2037] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:10.045786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.057320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42747b-31a8-4702-8bd1-8d7aa52eaa32 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.062895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.063514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.063865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.064146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.064884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:37:10.065226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.066817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.082744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1014/2038] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:37:10.084534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.094171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b1982-488b-4d1c-aa6a-0d20eec2fe91 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.099662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.100316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.100596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.101032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.101579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.103460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.117854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:10.229375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1025/2039] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:10.233654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.243914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d01b42-0ec5-42a1-a1db-809ed90909af admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.304935 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.305583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.305860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.306132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.306792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.308971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.419055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1015/2040] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:10.420214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.431897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0741a-6172-46c4-a872-451bcd014fe9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.436703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.437361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.437673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.437913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.438611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.440225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.461442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1026/2041] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:10.465279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.475766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3527d2-d171-4954-857b-ee938b5eb84a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.480457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.481108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.481396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.481673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.482376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.483980 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525d71-1899-4d49-a470-e517c3eb1600 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.495918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1016/2042] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:10.499181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.509858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51edc-8df4-42d1-b2e5-d57cab2275d6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.514526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.515235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.515455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.515732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] PATH_INFO: `/v3/projects/0a47a7efece84c2e9599af59706c5099/users/9dd58fee03714d2dab68e6882c835493/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.530851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a47a7efece84c2e9599af59706c5099, user_id=9dd58fee03714d2dab68e6882c835493, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.539055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.604139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1027/2043] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:10 2022] PUT /identity/v3/projects/0a47a7efece84c2e9599af59706c5099/users/9dd58fee03714d2dab68e6882c835493/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:10.616011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.616300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.616581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.644441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] MFA Rules not processed for user `9dd58fee03714d2dab68e6882c835493`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:10.645677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:10.686709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:10.709295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1017/2044] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:10.713743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.776174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e433bb-3505-46b0-ab58-a95b97928c68 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.780914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.781557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.781839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.782112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.782827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.784740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:10.868214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1028/2045] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:37:10.879295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:10.890061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821c3f2-bb40-49a3-956a-a375729fb2bc None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:10.946848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:10.947489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:10.947662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:10.947939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:10.948604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:10.950569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.069983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1018/2046] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:10 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:11.074339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:11.085425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d914508-a044-4cb6-b8d1-daf020e38c43 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:11.090662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:11.091270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.091552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.091829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.092533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:11.094162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.105760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1029/2047] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:11.114764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:11.126387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68005362-b84e-42fe-8eca-a7ab59893aa1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:11.132499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:11.133218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.133505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.133780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] PATH_INFO: `/v3/projects/0b8a14267e02405e9da2a46e62c3cea1/users/d653d4e4727248de9d23b158f1ea1209/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.151703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8a14267e02405e9da2a46e62c3cea1, user_id=d653d4e4727248de9d23b158f1ea1209, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:11.154423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.218302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1019/2048] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:11 2022] PUT /identity/v3/projects/0b8a14267e02405e9da2a46e62c3cea1/users/d653d4e4727248de9d23b158f1ea1209/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:11.223690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:11.234329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d2245-1c67-4bb5-98e0-ce26fae8b0c4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:11.291011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:11.291628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.291909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.292180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.292897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:11.294759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b557a-afd3-4288-9ae3-bd87dadc0824 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.308457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1030/2049] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:11.309720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:11.320363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5314-401f-4c28-be98-ecc9b48c114b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:11.325015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:11.325692 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.325972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.326250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] PATH_INFO: `/v3/projects/0b8a14267e02405e9da2a46e62c3cea1/users/d653d4e4727248de9d23b158f1ea1209/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.328177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8a14267e02405e9da2a46e62c3cea1, user_id=d653d4e4727248de9d23b158f1ea1209, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:11.330851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.342087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1020/2050] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:11 2022] PUT /identity/v3/projects/0b8a14267e02405e9da2a46e62c3cea1/users/d653d4e4727248de9d23b158f1ea1209/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:11.352528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.352814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.353125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.381935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] MFA Rules not processed for user `d653d4e4727248de9d23b158f1ea1209`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:11.383153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:11.396417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:11.458632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1031/2051] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:11.466829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:11.471713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:11.527473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a20d1-f0ec-42a0-a9ad-706564b2f36a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:11.531805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:11.532427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:11.532709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:11.533022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:11.584353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:11.595731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:11.597194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:11.607048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1021/2052] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:21.486391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=22 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:21.649183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:23.064103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:23.074911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42065c-9ef8-4ab0-9cca-a59b661211a7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:23.079016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:23.079637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:23.079915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:23.080226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:23.089753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:23.091182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:23.100960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1032/2053] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:25.897135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:25.899787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:26.005152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8c057-0519-4388-940b-9c1080ee2d5c service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:26.011063 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:26.011630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:26.011975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:26.012316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] PATH_INFO: `/v3/users/f489c80ce3324b818096ede43b690fb4` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:26.013675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f489c80ce3324b818096ede43b690fb4)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:26.015998 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:26.240523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f489c80ce3324b818096ede43b690fb4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.253681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f489c80ce3324b818096ede43b690fb4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.332724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f489c80ce3324b818096ede43b690fb4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.344253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f489c80ce3324b818096ede43b690fb4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.344253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a9fdde6d-65e7-47ba-a60f-cb745615f359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f489c80ce3324b818096ede43b690fb4'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.355707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1022/2054] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:25 2022] DELETE /identity/v3/users/f489c80ce3324b818096ede43b690fb4 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:26.660150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:26.673165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6599f16-ec07-4591-ba28-ad2692465054 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:26.747311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:26.747968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:26.748255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:26.748552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] PATH_INFO: `/v3/projects/930127f777e34dd1911ed0443282d41a` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:26.749757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=930127f777e34dd1911ed0443282d41a)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:26.753457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:26.887172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930127f777e34dd1911ed0443282d41a', 'request_id': 'req-b01a44a1-5c76-4a59-9fd2-280084e25806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:26.895033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930127f777e34dd1911ed0443282d41a', 'request_id': 'req-b01a44a1-5c76-4a59-9fd2-280084e25806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:27.067808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930127f777e34dd1911ed0443282d41a', 'request_id': 'req-b01a44a1-5c76-4a59-9fd2-280084e25806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:27.085333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] The token cache is being invalidate because project 930127f777e34dd1911ed0443282d41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:27.086268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:27.086999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-b01a44a1-5c76-4a59-9fd2-280084e25806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:27.088557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1033/2055] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:26 2022] DELETE /identity/v3/projects/930127f777e34dd1911ed0443282d41a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:27.094157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.094654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.094752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.119461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:27.120813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:27.189105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1023/2056] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:27.197763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7acd-822b-4200-853e-70131c09e36f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.198056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7acd-822b-4200-853e-70131c09e36f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.198333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7acd-822b-4200-853e-70131c09e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.218707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-424a7acd-822b-4200-853e-70131c09e36f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:27.219914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-424a7acd-822b-4200-853e-70131c09e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:27.228307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1034/2057] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:27.233839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.247091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e088d46-0b06-4553-817e-d43a1f17ed0b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.256315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.258366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.259014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.259570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.260883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:37:27.261714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.263708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.277053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1024/2058] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:37:27.281458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a7acd-822b-4200-853e-70131c09e36f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.290872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a7acd-822b-4200-853e-70131c09e36f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.295912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.296656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.296936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.297246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.297921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.299824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.409490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1035/2059] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:27 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:37:27.413699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.423812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c96c8-13a5-4465-b71f-72a4ff88e12a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.428361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:27.484747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.485476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.485977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.486499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.487702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.489754 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.588466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1025/2060] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:27 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:27.590833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.601548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ccdad-8d8a-4775-ab50-6e18279791af admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.606202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.606817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.607095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.607368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.608069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.609768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.620880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1036/2061] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:27.624898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.635921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be2d0-b7b4-4e58-9c29-a2c58ffe80ef admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.641185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.641806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.642091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.642371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.643069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.644696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128773a-9b40-4e5b-b635-03854f97d3a1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.656076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1026/2062] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:27.665765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.680107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b68712-23d2-475f-8324-22de639dacd0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:27.686920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:27.687849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.688317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.688667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] PATH_INFO: `/v3/projects/8b01ff17835d4bb4a4062552851fa8a3/users/f29a3aaf9e9d4f0393ec7f9a912f08a8/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.709034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b01ff17835d4bb4a4062552851fa8a3, user_id=f29a3aaf9e9d4f0393ec7f9a912f08a8, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:27.712324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:27.807623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1037/2063] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:27 2022] PUT /identity/v3/projects/8b01ff17835d4bb4a4062552851fa8a3/users/f29a3aaf9e9d4f0393ec7f9a912f08a8/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:27.818194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7949-aa0a-4823-979d-111a0c0290f7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:27.818502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7949-aa0a-4823-979d-111a0c0290f7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:27.818775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7949-aa0a-4823-979d-111a0c0290f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:27.846529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b1ec7949-aa0a-4823-979d-111a0c0290f7 None None] MFA Rules not processed for user `f29a3aaf9e9d4f0393ec7f9a912f08a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:27.847812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b1ec7949-aa0a-4823-979d-111a0c0290f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:27.956282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1027/2064] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:27.966951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:27.968637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:27.984101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:28.048835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863ec56-1288-4b1e-98cc-305b60af5670 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:28.055606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:28.056235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:28.056533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:28.056919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:28.068119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:28.069822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:28.079677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1038/2065] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:28.668850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13cfc-c94f-4bfb-99c8-5a5b1a5589c5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:28.669292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13cfc-c94f-4bfb-99c8-5a5b1a5589c5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:28.669292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13cfc-c94f-4bfb-99c8-5a5b1a5589c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:28.690143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bdc13cfc-c94f-4bfb-99c8-5a5b1a5589c5 None None] MFA Rules not processed for user `227203ab050f43f793d8c4fe3c31497d`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:28.691343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bdc13cfc-c94f-4bfb-99c8-5a5b1a5589c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:28.747449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1028/2066] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:30.303691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:30.306390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:30.400955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f330f-74eb-4c5a-a886-ea3c592fe6b5 service nova] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:30.410254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:30.410991 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:30.411333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:30.411789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] PATH_INFO: `/v3/users/9dd58fee03714d2dab68e6882c835493` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:30.413183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd58fee03714d2dab68e6882c835493)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:30.415900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:30.635392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd58fee03714d2dab68e6882c835493'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:30.654302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd58fee03714d2dab68e6882c835493'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:30.661266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd58fee03714d2dab68e6882c835493'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:30.662009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd58fee03714d2dab68e6882c835493'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:30.744203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd58fee03714d2dab68e6882c835493'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:30.750686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1039/2067] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:30 2022] DELETE /identity/v3/users/9dd58fee03714d2dab68e6882c835493 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:30.755233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:30.755744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:30.755837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:30.775675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:30.776866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:30.789763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:30.836810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1029/2068] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:30.847087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:30.848606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:30.870543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:30.907769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886425-aecb-4787-ab5c-dc92022a3180 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:30.921018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:30.921018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:30.921018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:30.921018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:30.926187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:30.927671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:30.939345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1040/2069] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:31.370026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:31.376037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:31.386916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf50e74-8f45-4931-b23f-b4f8cae265d3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:31.391705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:31.392359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:31.392639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:31.392915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] PATH_INFO: `/v3/projects/0a47a7efece84c2e9599af59706c5099` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:31.394203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a47a7efece84c2e9599af59706c5099)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:31.396128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:31.504861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a47a7efece84c2e9599af59706c5099', 'request_id': 'req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.601271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a47a7efece84c2e9599af59706c5099', 'request_id': 'req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.614966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a47a7efece84c2e9599af59706c5099', 'request_id': 'req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.626882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] The token cache is being invalidate because project 0a47a7efece84c2e9599af59706c5099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:31.627177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.627847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.629566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1030/2070] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:31 2022] DELETE /identity/v3/projects/0a47a7efece84c2e9599af59706c5099 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:31.633851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:31.644537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0081-207c-448e-9fb3-6570a55ad177 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:31.697565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:31.697565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:31.726171 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:31.728253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:31.729233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:31.729630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] PATH_INFO: `/v3/users/d653d4e4727248de9d23b158f1ea1209` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:31.731645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d653d4e4727248de9d23b158f1ea1209)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:31.735024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:31.770565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea594-3fc6-4ad5-94ab-7a043d030a84 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:31.776769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:31.777535 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:31.777863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:31.778303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] PATH_INFO: `/v3/users/f29a3aaf9e9d4f0393ec7f9a912f08a8` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:31.779499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29a3aaf9e9d4f0393ec7f9a912f08a8)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:31.781466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:31.975988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd653d4e4727248de9d23b158f1ea1209'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:31.990986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd653d4e4727248de9d23b158f1ea1209'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.016840 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd653d4e4727248de9d23b158f1ea1209'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.017917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd653d4e4727248de9d23b158f1ea1209'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.031581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29a3aaf9e9d4f0393ec7f9a912f08a8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.033406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:32.047316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29a3aaf9e9d4f0393ec7f9a912f08a8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.095344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cbca755f-56a8-45f6-8882-ec65dd78743a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd653d4e4727248de9d23b158f1ea1209'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.105565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1041/2071] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:31 2022] DELETE /identity/v3/users/d653d4e4727248de9d23b158f1ea1209 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:32.186580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29a3aaf9e9d4f0393ec7f9a912f08a8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.196401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29a3aaf9e9d4f0393ec7f9a912f08a8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.197570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29a3aaf9e9d4f0393ec7f9a912f08a8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.208220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1031/2072] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:31 2022] DELETE /identity/v3/users/f29a3aaf9e9d4f0393ec7f9a912f08a8 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:32.213797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:32.214066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:32.214416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:32.251361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:32.252576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:32.289406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:32.317100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1042/2073] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:32.327632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:32.329148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:32.389875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a9204-7f5c-4020-aefb-614c51c514a1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:32.394025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:32.394751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:32.395104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:32.395451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:32.405679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:32.407120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab266bf-e9a2-43c6-9c40-d019321e157a service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:32.417901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1032/2074] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:32.586483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:32.588528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:32.606950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171ae0-8024-4a79-97e4-abbb3d15508f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:32.613422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:32.614287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:32.616019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:32.616921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] PATH_INFO: `/v3/projects/0b8a14267e02405e9da2a46e62c3cea1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:32.619191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8a14267e02405e9da2a46e62c3cea1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:32.621684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:32.738394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8a14267e02405e9da2a46e62c3cea1', 'request_id': 'req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.744688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8a14267e02405e9da2a46e62c3cea1', 'request_id': 'req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.847185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8a14267e02405e9da2a46e62c3cea1', 'request_id': 'req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.860890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] The token cache is being invalidate because project 0b8a14267e02405e9da2a46e62c3cea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:32.861226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.862045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-845dc973-1ce6-4b1a-b6a5-190f156d8ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:32.870789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1043/2075] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:32 2022] DELETE /identity/v3/projects/0b8a14267e02405e9da2a46e62c3cea1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:32.873697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:32.874210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:32.878569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:32.900620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:32.905223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:32.979514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1033/2076] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:32.989323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:32.989607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:32.989879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.003054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.004710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.026822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7365b3c-2ed1-48a4-9499-18acfad0aa63 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.032743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:33.032743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.047951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.050035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.050370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.050670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] PATH_INFO: `/v3/projects/8b01ff17835d4bb4a4062552851fa8a3` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.052179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b01ff17835d4bb4a4062552851fa8a3)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.057535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1044/2077] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:33.057815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.060199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.071856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed489b-0127-4990-a220-6bd3d590f6b4 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.078149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.078861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.079315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.079653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.080489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:37:33.080862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.082574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.095705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1045/2078] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:37:33.100031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.109935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435208a-c1cc-4711-910a-2438188adbe1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.114470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.115054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.115348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.115613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.116281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.118807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.208261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b01ff17835d4bb4a4062552851fa8a3', 'request_id': 'req-90e51557-c531-4632-beb7-f3fbfda30bf6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:33.299977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1046/2079] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:37:33.304075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.305035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b01ff17835d4bb4a4062552851fa8a3', 'request_id': 'req-90e51557-c531-4632-beb7-f3fbfda30bf6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:33.312448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b01ff17835d4bb4a4062552851fa8a3', 'request_id': 'req-90e51557-c531-4632-beb7-f3fbfda30bf6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:33.313774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:33.324246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] The token cache is being invalidate because project 8b01ff17835d4bb4a4062552851fa8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:33.324544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:33.325249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-90e51557-c531-4632-beb7-f3fbfda30bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:33.326796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1034/2080] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:33 2022] DELETE /identity/v3/projects/8b01ff17835d4bb4a4062552851fa8a3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:33.333528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f023-c997-473e-878a-6432a56d345d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.333815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f023-c997-473e-878a-6432a56d345d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.334101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f023-c997-473e-878a-6432a56d345d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.364081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bdc6f023-c997-473e-878a-6432a56d345d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:33.365372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bdc6f023-c997-473e-878a-6432a56d345d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.374172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a66f-232d-430b-bc58-88a219f1c502 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.374993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1035/2081] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:33.378970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.379591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.379878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.380154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.380820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.383590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.384869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.385204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.385481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.405641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:33.407243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.416921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1036/2082] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:33.421181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.430679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1079-9156-47c6-9f68-922c2e185fd2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.435741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.436338 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.436624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.436901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.437686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:37:33.437992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.439613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.451378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1037/2083] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:37:33.455686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.466522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0d23b-c57d-48d5-962b-152ad1e35694 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.471949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.472636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.472917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.473279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.474060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.476702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.499694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1047/2084] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:37:33.503114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.504835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.517182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fdba-5d1a-4faa-bdae-a8354bbd9023 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.521775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.522363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.522645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.522918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.523683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.525306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.530438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:33.535264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1048/2085] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:37:33.539730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.548864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ae00-a8ad-4ec2-b1f7-71b8775153d0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.553551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.554124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.554987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.555266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.555987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.557616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.568417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1049/2086] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:37:33.572401 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.581836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935773-83fc-4218-97fd-9b1ae60e923a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.585551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1038/2087] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:37:33.589576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.589781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.590433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.590714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.590993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] PATH_INFO: `/v3/projects/a8d79958e09b49c7afbd02b8daee6717/users/ecb4773a6ba2450181d9a10c0cfbbb21/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.607082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d79958e09b49c7afbd02b8daee6717, user_id=ecb4773a6ba2450181d9a10c0cfbbb21, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.609766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38befa87-a1fb-4396-82d2-48e1dade0eea admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.650553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5a1e-127e-4c1d-9b73-330a728b5a3c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.655276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.655888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.656168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.656444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.657165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.659238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.676960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1050/2088] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:33 2022] PUT /identity/v3/projects/a8d79958e09b49c7afbd02b8daee6717/users/ecb4773a6ba2450181d9a10c0cfbbb21/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:33.689652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.689942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.690221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.717512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] MFA Rules not processed for user `ecb4773a6ba2450181d9a10c0cfbbb21`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:33.718704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.782752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1051/2089] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:37:33.787192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1039/2090] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:33 2022] POST /identity/v3/users => generated 282 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:33.792875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.795106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.796859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:33.855122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ae920-e1e8-4ef0-bb11-e6a9a9540f69 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.859541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd697-d183-41c4-967d-77b84ac92b04 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.869708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.870486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.871092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.871680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.872613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.875011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.894403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1052/2091] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:33.896706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.907056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a827917-20fd-46aa-93d8-e624814cd251 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.907211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:33.911805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.912352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.912630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.912904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.913045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.913605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.913745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.914061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.914463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.915850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.923273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.924777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bda30-2a18-493b-83ef-47ee99d9db19 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:33.929133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1053/2092] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:33.931160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:33.935252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1040/2093] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:37:33.941197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84297d6-24c9-4991-9339-34f9f068401e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:33.947040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:33.947770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:33.948058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:33.948331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] PATH_INFO: `/v3/projects/d77c7768b3b346848da14ab56b0fa5f2/users/3d59527c73984b66a88a714e4182ffcf/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:33.964083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77c7768b3b346848da14ab56b0fa5f2, user_id=3d59527c73984b66a88a714e4182ffcf, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:33.966781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.053532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1054/2094] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:33 2022] PUT /identity/v3/projects/d77c7768b3b346848da14ab56b0fa5f2/users/3d59527c73984b66a88a714e4182ffcf/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:34.066765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.067130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.067481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.101073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] MFA Rules not processed for user `3d59527c73984b66a88a714e4182ffcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:34.102560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:34.186849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.188077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1041/2095] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:34.212963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:34.281750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959a3d-f3a9-4c13-a10d-e084223aeaf6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.296241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.296888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.297257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.297944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.307627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.307627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.383272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1055/2096] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:34 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:34.387381 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.398733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe0aba-f955-4305-b2bf-810601962260 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.472706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.473380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.473662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.473938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.474622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.476689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.581982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1042/2097] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:34 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:34.586507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.598621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d285035-0c09-4fb9-b8d2-20ac2f63c25c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.603881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.604519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.604800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.605113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.606225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.608347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.621307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1056/2098] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:34.625729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.641420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d6d12-80ca-45e8-bad8-1505a90eb164 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.648218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.648218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.648218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.648218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] PATH_INFO: `/v3/projects/60940806b7a74b03bf2a39d648b8ddec/users/f9ff50d90c344aa2a18b200402c9ac0f/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.667001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60940806b7a74b03bf2a39d648b8ddec, user_id=f9ff50d90c344aa2a18b200402c9ac0f, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.669883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.749420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1043/2099] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:34 2022] PUT /identity/v3/projects/60940806b7a74b03bf2a39d648b8ddec/users/f9ff50d90c344aa2a18b200402c9ac0f/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:34.753511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.764678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c412d3-fdf9-4b2b-b673-a3789ee28641 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.828516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.829215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.829509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.829786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.830531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.832124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924b4f2-1241-42e7-a90e-31866b7596c8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.842828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1057/2100] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:34.847238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:34.860648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871023cb-3299-4adf-88d5-c568d9259426 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:34.865636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:34.866307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.866614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.867596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] PATH_INFO: `/v3/projects/60940806b7a74b03bf2a39d648b8ddec/users/f9ff50d90c344aa2a18b200402c9ac0f/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.869799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] RBAC: Authorizing `identity:create_grant(project_id=60940806b7a74b03bf2a39d648b8ddec, user_id=f9ff50d90c344aa2a18b200402c9ac0f, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:34.872624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:34.884675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1044/2101] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:34 2022] PUT /identity/v3/projects/60940806b7a74b03bf2a39d648b8ddec/users/f9ff50d90c344aa2a18b200402c9ac0f/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:34.894933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:34.895299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:34.895646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:34.931348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] MFA Rules not processed for user `f9ff50d90c344aa2a18b200402c9ac0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:34.932723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:35.006954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1058/2102] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:35.020844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:35.055856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:35.080923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f41e1b-a9f5-42e1-973f-74f8dd9a37da admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:35.084632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:35.085277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:35.086155 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:35.086907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:35.096350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:35.097816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc6e6e-8ab4-4221-b0cd-72ecf7ec4e22 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:35.107356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1045/2103] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:36.044355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:36.057596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:36.108747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a868a-c92d-4496-81f3-800f8bef8994 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:36.113388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:36.114026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:36.114304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:36.114576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] PATH_INFO: `/v3/users/3d59527c73984b66a88a714e4182ffcf` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:36.115723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d59527c73984b66a88a714e4182ffcf)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:36.118701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:36.272310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d59527c73984b66a88a714e4182ffcf'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:36.284610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d59527c73984b66a88a714e4182ffcf'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:36.300459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d59527c73984b66a88a714e4182ffcf'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:36.300459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d59527c73984b66a88a714e4182ffcf'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:36.380572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d59527c73984b66a88a714e4182ffcf'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:36.387927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1059/2104] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:36 2022] DELETE /identity/v3/users/3d59527c73984b66a88a714e4182ffcf => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:36.392477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:36.392928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:36.393049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:36.411606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:36.412802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:36.495677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1046/2105] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:36.509467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:36.510994 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:36.586657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848ec0-4f3b-4be6-b3d9-5566462a7b29 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:36.590656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:36.591469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:36.591766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:36.592030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:36.610798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:36.613162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:36.637861 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1060/2106] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:37:37.149366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:37.157330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:37.181562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d39d6c-b712-4317-88db-1035c46cd1ff None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:37.196544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:37.196544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:37.196544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:37.196544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] PATH_INFO: `/v3/projects/d77c7768b3b346848da14ab56b0fa5f2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:37.196918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77c7768b3b346848da14ab56b0fa5f2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:37.196918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:37.384220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77c7768b3b346848da14ab56b0fa5f2', 'request_id': 'req-a0e63b06-1956-4e51-8a85-893c23d2ee9b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.463892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77c7768b3b346848da14ab56b0fa5f2', 'request_id': 'req-a0e63b06-1956-4e51-8a85-893c23d2ee9b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.470661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77c7768b3b346848da14ab56b0fa5f2', 'request_id': 'req-a0e63b06-1956-4e51-8a85-893c23d2ee9b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.480941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] The token cache is being invalidate because project d77c7768b3b346848da14ab56b0fa5f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:37.481257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.481885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.483418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1047/2107] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:37 2022] DELETE /identity/v3/projects/d77c7768b3b346848da14ab56b0fa5f2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:37.487469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:37.496937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531212d-2464-49fa-9e47-ce5060e87bfc service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:37.527659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:37.561471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:37.562134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:37.562426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:37.562822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] PATH_INFO: `/v3/users/f9ff50d90c344aa2a18b200402c9ac0f` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:37.563917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ff50d90c344aa2a18b200402c9ac0f)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:37.565961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:37.760004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ff50d90c344aa2a18b200402c9ac0f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.775274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ff50d90c344aa2a18b200402c9ac0f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.784005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ff50d90c344aa2a18b200402c9ac0f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.785116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ff50d90c344aa2a18b200402c9ac0f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.852712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6a9984b5-8397-414b-8022-deca2c398a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ff50d90c344aa2a18b200402c9ac0f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:37.860628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1061/2108] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:37:37 2022] DELETE /identity/v3/users/f9ff50d90c344aa2a18b200402c9ac0f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:38.452058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:38.453900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:38.471234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e63b06-1956-4e51-8a85-893c23d2ee9b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:38.527621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:38.533961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:38.534602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:38.534884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:38.535314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] PATH_INFO: `/v3/projects/60940806b7a74b03bf2a39d648b8ddec` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:38.537165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60940806b7a74b03bf2a39d648b8ddec)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:38.541519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:38.638972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60940806b7a74b03bf2a39d648b8ddec', 'request_id': 'req-db977720-de6b-4ca3-acb1-ea0db2dd95a5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:38.718119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60940806b7a74b03bf2a39d648b8ddec', 'request_id': 'req-db977720-de6b-4ca3-acb1-ea0db2dd95a5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:38.725041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60940806b7a74b03bf2a39d648b8ddec', 'request_id': 'req-db977720-de6b-4ca3-acb1-ea0db2dd95a5'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:38.736458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] The token cache is being invalidate because project 60940806b7a74b03bf2a39d648b8ddec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:37:38.736742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:38.737387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-db977720-de6b-4ca3-acb1-ea0db2dd95a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:37:38.739207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1048/2109] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:37:38 2022] DELETE /identity/v3/projects/60940806b7a74b03bf2a39d648b8ddec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:37:38.745179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:38.745625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:38.745884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:38.766156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:38.767344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:38.829593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1062/2110] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:38.842600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:38.842890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:38.843433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:38.864325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:38.865681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:38.874323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1049/2111] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:38.878638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:38.888680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518f1d2-49fd-446b-8369-aa7ea9927f40 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:38.893280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:38.893890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:38.894172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:38.894447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:38.895145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:37:38.895450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:38.897066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:38.908691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1063/2112] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:37:38.912887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:38.922769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb65dd-5186-485e-b93f-4e3cb69e20db None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:38.927314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:38.927942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:38.928221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:38.928882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:38.929184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:38.931743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.022788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1050/2113] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:38 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:39.028032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.038540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b7f13-999e-4084-bfe3-b4c88d199a40 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.079825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:39.099310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.099921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.100266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.100547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.101357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.103584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.198974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1064/2114] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:39 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:39.203044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.213353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39174bf2-daf8-47cf-bde0-fa65a1ff67ba admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.217946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.218549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.218830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.219111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.219814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.221433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.231517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1051/2115] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:39.235210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.246427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2f583-38ed-44f0-8663-c6d90480992e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.250867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.251466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.251744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.252012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.252698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.254305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacfbc6-afda-4cab-8579-bd66212aea8a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.265537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1065/2116] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:39.269490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.279521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e851f2-ef98-46d4-8f5d-ddc767414039 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.284273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.284927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.285240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.285517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] PATH_INFO: `/v3/projects/dcdea18860ed40d4949705e4a5a37f03/users/c9925a5de0f04c0d88c8cc3b237a39e1/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.299967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdea18860ed40d4949705e4a5a37f03, user_id=c9925a5de0f04c0d88c8cc3b237a39e1, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.302648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.367513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1052/2117] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:39 2022] PUT /identity/v3/projects/dcdea18860ed40d4949705e4a5a37f03/users/c9925a5de0f04c0d88c8cc3b237a39e1/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:39.377671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.377988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.378258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.406337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] MFA Rules not processed for user `c9925a5de0f04c0d88c8cc3b237a39e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:39.407567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:39.471881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1066/2118] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:39.476771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.541807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9d48b-031b-4a87-b78e-1434f863ec71 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.546527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.547142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.547425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.547703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.548375 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.550298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.662150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1053/2119] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:37:39 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:39.666502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.677064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bbfbc-b4f9-4394-8922-c2f14f0b637d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.733578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.734184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.734460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.734729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.735385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.737306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.824865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1067/2120] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:37:39 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:37:39.828835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.839246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997a123-042a-44ae-93bf-55c379b96bfd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.843844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.844452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.844737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.845044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.845755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.847348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.849458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:39.858279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1054/2121] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:37:39.862489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.873207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe261-75ee-4554-be21-bc3da3af82c9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:39.877745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:39.878396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:39.878678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:39.878950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] PATH_INFO: `/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e/users/9a375e1362c241dabe6b4f9f9a3d2b0c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:39.894456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab5dc2dd89e437e9e75eba1251f8b5e, user_id=9a375e1362c241dabe6b4f9f9a3d2b0c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:39.897267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:39.972487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1068/2122] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:39 2022] PUT /identity/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e/users/9a375e1362c241dabe6b4f9f9a3d2b0c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:39.977223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:39.987205 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43710e1-e384-40a8-9c4d-4d9b202c3415 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:40.042624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:40.043237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:40.043516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:40.043794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:40.044486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:40.046102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f1c29-3bf9-41b7-a290-2093dd48c1fa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:40.056635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1055/2123] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:37:40.061181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:40.072403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896da994-2cf2-4f07-8e8e-0bf17d40cdb6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:40.077002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:40.077666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:40.077941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:40.078212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] PATH_INFO: `/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e/users/9a375e1362c241dabe6b4f9f9a3d2b0c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:40.080160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab5dc2dd89e437e9e75eba1251f8b5e, user_id=9a375e1362c241dabe6b4f9f9a3d2b0c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:40.082863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:40.089942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:40.094326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1069/2124] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:37:40 2022] PUT /identity/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e/users/9a375e1362c241dabe6b4f9f9a3d2b0c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:37:40.106108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:40.106396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:40.106676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:40.135226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] MFA Rules not processed for user `9a375e1362c241dabe6b4f9f9a3d2b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:37:40.136445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:40.210476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1056/2125] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:37:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:37:40.221344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:40.223029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:37:40.284713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70222-6a45-46e9-aaf9-37ad0c5416ae admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:40.288548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:40.289747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:40.290030 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:40.290307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:40.353819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:40.355297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:40.366073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1070/2126] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:37:50.207418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=64 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:50.361717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=48 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:37:58.110712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:37:58.122199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a5949-f8c1-4ecc-a48e-631c09d1b1cd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:37:58.127508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:37:58.128144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:37:58.128423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:37:58.128695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:37:58.141093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:37:58.142315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88acec7-a4ef-44c6-a492-980d6df7af20 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:37:58.152331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1057/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:38:08.151628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:20.837695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:20.839284 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:20.906272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6e96-bac1-4c4d-b647-cc7a057cf2ed service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:20.910815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:20.911460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:20.911754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:20.912029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] PATH_INFO: `/v3/users/ecb4773a6ba2450181d9a10c0cfbbb21` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:20.913241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb4773a6ba2450181d9a10c0cfbbb21)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:20.915190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:21.122094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb4773a6ba2450181d9a10c0cfbbb21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:21.133814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb4773a6ba2450181d9a10c0cfbbb21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:21.140714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb4773a6ba2450181d9a10c0cfbbb21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:21.141498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb4773a6ba2450181d9a10c0cfbbb21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:21.222348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb4773a6ba2450181d9a10c0cfbbb21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:21.228768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1071/2128] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:38:20 2022] DELETE /identity/v3/users/ecb4773a6ba2450181d9a10c0cfbbb21 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:21.233948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:21.238666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:21.238666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:21.254311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:21.255521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:21.314029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1058/2129] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:21.324377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:21.325796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:21.381957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508f87-eea6-4da6-90b5-a1b5e407fb1e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:21.386130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:21.386733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:21.387014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:21.387289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:21.396117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:21.397568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74bf95-2e3b-43a5-bcb3-4db0b019a08c service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:21.407557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1072/2130] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:38:21.801482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:21.805077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:21.818585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0e0be-28b2-48ba-bb08-9a857b50f221 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:21.827786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:21.828501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:21.828784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:21.829089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] PATH_INFO: `/v3/projects/a8d79958e09b49c7afbd02b8daee6717` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:21.830557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d79958e09b49c7afbd02b8daee6717)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:21.833022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:21.953271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d79958e09b49c7afbd02b8daee6717', 'request_id': 'req-3a365e88-ffd5-474b-a38b-2e52f43e33d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:22.057884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d79958e09b49c7afbd02b8daee6717', 'request_id': 'req-3a365e88-ffd5-474b-a38b-2e52f43e33d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:22.066040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d79958e09b49c7afbd02b8daee6717', 'request_id': 'req-3a365e88-ffd5-474b-a38b-2e52f43e33d6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:22.077526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] The token cache is being invalidate because project a8d79958e09b49c7afbd02b8daee6717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:38:22.077808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:22.078489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:22.080006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1059/2131] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:38:21 2022] DELETE /identity/v3/projects/a8d79958e09b49c7afbd02b8daee6717 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:31.403431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:32.074451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:47.620053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:47.620569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:47.620657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:47.641334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:47.642535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:47.721135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1073/2132] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:47.765969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:47.767555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:47.788617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:47.793969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:47.841941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365e88-ffd5-474b-a38b-2e52f43e33d6 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:47.847061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:47.847691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:47.847970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:47.848420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:47.860162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:47.861923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0cbeb-0fdc-40e8-8f0a-1b402cfbed3c service swift] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:47.868425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284464a-7045-4d1a-90ba-cf8a4ca5d9fa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:47.869696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:47.870547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:47.870825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:47.871120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] PATH_INFO: `/v3/users/a1fd257882f5457d8407383a926871f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:47.871572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1060/2133] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:38:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 14:38:47.872310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fd257882f5457d8407383a926871f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:47.874319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:48.104971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fd257882f5457d8407383a926871f1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.117709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fd257882f5457d8407383a926871f1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.125530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fd257882f5457d8407383a926871f1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.133326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fd257882f5457d8407383a926871f1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.209690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fd257882f5457d8407383a926871f1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.216692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1074/2134] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:38:47 2022] DELETE /identity/v3/users/a1fd257882f5457d8407383a926871f1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:38:48.221283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:48.221576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:48.221858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:48.241394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:48.242588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:48.299928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1061/2135] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:48.309928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:48.311320 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:48.346862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:48.363975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:48.365517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:48.378769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffadd9-8148-4002-a7c1-4da786508919 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:48.383883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:48.384443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:48.384637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:48.386605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] PATH_INFO: `/v3/users/c9925a5de0f04c0d88c8cc3b237a39e1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:48.387385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9925a5de0f04c0d88c8cc3b237a39e1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:48.389536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:48.392975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ff644-3a31-41c5-aa53-940996959c48 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:48.396857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:48.397530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:48.397858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:48.398308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:48.411326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:48.414012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff46c6-2f1b-4963-b454-e27654063058 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:48.444660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1075/2136] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:38:48.580957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9925a5de0f04c0d88c8cc3b237a39e1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.596866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9925a5de0f04c0d88c8cc3b237a39e1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.667811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9925a5de0f04c0d88c8cc3b237a39e1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.675684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9925a5de0f04c0d88c8cc3b237a39e1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.677568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9925a5de0f04c0d88c8cc3b237a39e1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:48.702817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1062/2137] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:38:48 2022] DELETE /identity/v3/users/c9925a5de0f04c0d88c8cc3b237a39e1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:48.707423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:48.708376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:48.708821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:48.738475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:48.740068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:48.827335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1076/2138] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:48.841963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:48.843811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:48.849822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:48.923345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b15f0b-0011-4834-adf7-a351b19b1dcb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:48.927278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:48.927913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:48.928195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:48.928685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:48.939884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:48.941410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:48.952705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1063/2139] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:38:48.976727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:48.978444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:49.000906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfecc0b-806a-45ba-bc2b-d5de96b48748 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:49.007193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:49.008234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:49.008577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:49.009104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] PATH_INFO: `/v3/projects/6cefe8f925be47f0b74012553230aa46` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:49.016752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cefe8f925be47f0b74012553230aa46)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:49.016752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:49.182109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cefe8f925be47f0b74012553230aa46', 'request_id': 'req-7386ff75-a6b0-4054-b217-4be35b136e6e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:49.190225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cefe8f925be47f0b74012553230aa46', 'request_id': 'req-7386ff75-a6b0-4054-b217-4be35b136e6e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:49.284487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cefe8f925be47f0b74012553230aa46', 'request_id': 'req-7386ff75-a6b0-4054-b217-4be35b136e6e'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:49.299483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] The token cache is being invalidate because project 6cefe8f925be47f0b74012553230aa46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:38:49.299913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:49.300633 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:49.302439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1077/2140] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:38:48 2022] DELETE /identity/v3/projects/6cefe8f925be47f0b74012553230aa46 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:49.773704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:49.775331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:49.793732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89fbeb-e6a2-4277-b871-32789ce9959f service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:49.874777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:49.875661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:49.875789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:49.876069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] PATH_INFO: `/v3/projects/dcdea18860ed40d4949705e4a5a37f03` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:49.878284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcdea18860ed40d4949705e4a5a37f03)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:49.882478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:50.052307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcdea18860ed40d4949705e4a5a37f03', 'request_id': 'req-24df0e78-8c73-4b5c-878e-c1f20bc811f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.124527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcdea18860ed40d4949705e4a5a37f03', 'request_id': 'req-24df0e78-8c73-4b5c-878e-c1f20bc811f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.137943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcdea18860ed40d4949705e4a5a37f03', 'request_id': 'req-24df0e78-8c73-4b5c-878e-c1f20bc811f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.157285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] The token cache is being invalidate because project dcdea18860ed40d4949705e4a5a37f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:38:50.157415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.158296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.163654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1064/2141] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:38:49 2022] DELETE /identity/v3/projects/dcdea18860ed40d4949705e4a5a37f03 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:50.164844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:50.166685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:50.180658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386ff75-a6b0-4054-b217-4be35b136e6e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:50.248711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:50.263611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:50.264407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:50.264695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:50.264968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] PATH_INFO: `/v3/users/9a375e1362c241dabe6b4f9f9a3d2b0c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:50.266250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a375e1362c241dabe6b4f9f9a3d2b0c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:50.268378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:50.443641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a375e1362c241dabe6b4f9f9a3d2b0c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.457981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a375e1362c241dabe6b4f9f9a3d2b0c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.466635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a375e1362c241dabe6b4f9f9a3d2b0c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.467420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a375e1362c241dabe6b4f9f9a3d2b0c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.542341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-d68350bb-4b80-4c6e-b049-8557ff813265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a375e1362c241dabe6b4f9f9a3d2b0c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:50.552360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1078/2142] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:38:50 2022] DELETE /identity/v3/users/9a375e1362c241dabe6b4f9f9a3d2b0c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:51.049808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:51.054712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:51.088421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df0e78-8c73-4b5c-878e-c1f20bc811f7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:51.136060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:51.166093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:51.167485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.168046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.168874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] PATH_INFO: `/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.169971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab5dc2dd89e437e9e75eba1251f8b5e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:51.172357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:51.290044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab5dc2dd89e437e9e75eba1251f8b5e', 'request_id': 'req-4d7ef552-aa5c-4f72-b5c8-74856b27c844'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:51.355846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab5dc2dd89e437e9e75eba1251f8b5e', 'request_id': 'req-4d7ef552-aa5c-4f72-b5c8-74856b27c844'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:51.363811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab5dc2dd89e437e9e75eba1251f8b5e', 'request_id': 'req-4d7ef552-aa5c-4f72-b5c8-74856b27c844'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:51.376922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] The token cache is being invalidate because project 0ab5dc2dd89e437e9e75eba1251f8b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:38:51.378136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:51.378136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4d7ef552-aa5c-4f72-b5c8-74856b27c844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:38:51.379558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1065/2143] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:38:51 2022] DELETE /identity/v3/projects/0ab5dc2dd89e437e9e75eba1251f8b5e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:51.384498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.385705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.385705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.406912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:51.408203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:51.534088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.538784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.538784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.539019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1079/2144] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:51.560516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:51.562110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:51.572084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1066/2145] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:51.576649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:51.586973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e43df-3dfb-4ba3-9502-d7e8aa33ca9c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:51.592623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:51.593314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.605048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.605048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.605048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:38:51.605048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:51.605048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:51.614205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1080/2146] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:38:51.614608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:51.636576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f709-d9a0-4a88-8846-b0be9ebb0ff8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:51.641892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:51.642529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.642815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.643094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.643774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:51.645765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:51.732218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1067/2147] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:38:51 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:38:51.737536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:51.750703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265570-6338-4c63-97f6-2e5dccaaac96 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:51.817785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:51.825419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:51.826223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.828766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.828766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.828766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:51.830606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:51.947792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1081/2148] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:38:51 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:38:51.951171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:51.974552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba68f15-d43b-4a85-a326-722e8a62beb1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:51.988172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:51.997199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:51.997199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:51.997199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:51.997199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:51.997199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.013883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1068/2149] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:38:52.015831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.032538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fff-261b-4669-bdd0-da36783d7a12 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.038485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.039241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.039692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.040116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.043634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.046439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff352e5b-90f8-45ac-96c5-40e4ef058ece admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.068061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1082/2150] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:38:52.069292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.081850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0128fa-4bf8-4b4d-b573-6829f80170b7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.087256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.089769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.089769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.089769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] PATH_INFO: `/v3/projects/e24142d3aba84f09adf0def2b598df40/users/27c1b20f3ff945bf98e7e49f83bef79c/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.115994 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24142d3aba84f09adf0def2b598df40, user_id=27c1b20f3ff945bf98e7e49f83bef79c, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.119346 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.214721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1069/2151] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:38:52 2022] PUT /identity/v3/projects/e24142d3aba84f09adf0def2b598df40/users/27c1b20f3ff945bf98e7e49f83bef79c/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:38:52.226372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.226783 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.227132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.258701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] MFA Rules not processed for user `27c1b20f3ff945bf98e7e49f83bef79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:52.260801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:52.352556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1083/2152] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:52.354180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.446742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8c1e6-98e8-4ff4-be3a-89ae8c076d9c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.453386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.454560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.455133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.455621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.457851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.460182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.469818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:52.568688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.579879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1070/2153] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:38:52 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:38:52.583945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab9d82-b442-41f6-9498-c3f663b8ec34 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.664644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.665083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.665504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.665787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.666537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.668481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.797858 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1084/2154] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:38:52 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:38:52.799407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.814446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf69e61-1d09-43bf-a498-7e033d5e0550 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.823163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.826403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.826403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.826403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.826403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.828224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.840557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1071/2155] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:38:52.844443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.855441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da67d43-b566-4b57-8309-b1ffa0d6eb02 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.860564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.861311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.861734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.862018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.862971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.864698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306191f3-63d8-4c53-ba82-70e55db6d8a3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:52.892793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1085/2156] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:38:52.894839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:52.906324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735a77-c438-4323-a626-aaf5f985502d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:52.912852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:52.920114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:52.920114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:52.920114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] PATH_INFO: `/v3/projects/2b35ae2b19894dad8c39120f0b2af452/users/f344edf6eb094581a01560b793d65941/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:52.936370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b35ae2b19894dad8c39120f0b2af452, user_id=f344edf6eb094581a01560b793d65941, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:52.939326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.058992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1072/2157] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:38:52 2022] PUT /identity/v3/projects/2b35ae2b19894dad8c39120f0b2af452/users/f344edf6eb094581a01560b793d65941/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:38:53.069817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.070243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.071200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.109828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] MFA Rules not processed for user `f344edf6eb094581a01560b793d65941`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:53.111295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:53.162349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:53.203740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1086/2158] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:38:53.208331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.314226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e782b6-483f-40a1-b81a-4dffa04bbd4a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:53.319924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:53.325206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.325206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.325206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.325206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:53.325536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.411771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1073/2159] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:38:53 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:38:53.415799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.431782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1ec74-fefa-435c-986e-adc454449c0c None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:53.507305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:53.507931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.508211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.508482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.509255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:53.512077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d91db-12d1-4c31-ba1d-1a6587a641cc admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.626643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1087/2160] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:38:53 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:38:53.630473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.649749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5441cac-7440-48be-a991-db640d9020b8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:53.656457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:53.657220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.657556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.657819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.658732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:53.661856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.669618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053335b-c241-47a2-8f88-353eb791913d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.669962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053335b-c241-47a2-8f88-353eb791913d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.670181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053335b-c241-47a2-8f88-353eb791913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.684477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1074/2161] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:38:53.686678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.699991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc5baa-0c5a-4959-b197-851cb79f5920 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:53.703747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d053335b-c241-47a2-8f88-353eb791913d None None] MFA Rules not processed for user `f4ae3fb6a7954a24b88df4f35f55020c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:53.708865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d053335b-c241-47a2-8f88-353eb791913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:53.717156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:53.717995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.718177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.718413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] PATH_INFO: `/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1/users/d7a10efefb714a26880d7f8ac0ebc7f8/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.738657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2892cd8aaa4bc19812dff1d5dbf8f1, user_id=d7a10efefb714a26880d7f8ac0ebc7f8, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:53.751943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.790329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1088/2162] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:38:53.836164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1075/2163] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:38:53 2022] PUT /identity/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1/users/d7a10efefb714a26880d7f8ac0ebc7f8/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:38:53.837828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053335b-c241-47a2-8f88-353eb791913d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.863790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053335b-c241-47a2-8f88-353eb791913d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:53.945953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:53.946685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:53.946971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:53.947292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:53.948012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:53.949663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6e6ff-5765-41bd-9070-55248b993fa2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:53.960107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1089/2164] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:38:53.964928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:53.977383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd9c2-c226-4c75-8b13-06ef367012d2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:54.001326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:54.001946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:54.002231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:54.002520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] PATH_INFO: `/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1/users/d7a10efefb714a26880d7f8ac0ebc7f8/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:54.004538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2892cd8aaa4bc19812dff1d5dbf8f1, user_id=d7a10efefb714a26880d7f8ac0ebc7f8, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:54.007629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:54.018923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1076/2165] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:38:53 2022] PUT /identity/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1/users/d7a10efefb714a26880d7f8ac0ebc7f8/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:38:54.029877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:54.030167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:54.030446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:54.039282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:54.073159 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] MFA Rules not processed for user `d7a10efefb714a26880d7f8ac0ebc7f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:38:54.074459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:54.169638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1090/2166] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:38:54.188201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:54.189953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:54.254695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:38:54.262323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583952-567d-4480-82c8-5a66ca21b507 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:54.267667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:54.273325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:54.273325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:54.273325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:54.282899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:54.284923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:54.298849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1077/2167] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:38:55.443751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:55.445363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:55.458176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7f9ca-9872-4b8c-81e0-5b69aa7e109e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:55.462091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:55.463279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:55.463565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:55.463978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:55.543100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:55.544565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:55.554353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1091/2168] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:38:57.192789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:38:57.195108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:57.287558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63670f3-d7da-42ee-a1ac-279fb2ee8129 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:38:57.293444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:38:57.294084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:38:57.294373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:38:57.294669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:38:57.296477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:38:57.309514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:38:57.311115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eec9e3-9939-4c5e-a752-c56d1aaa04e5 service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:38:57.323950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1078/2169] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:38:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:03.172745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:03.182754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584aa7c-ddbb-45de-ad8c-4bbd895d0929 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:03.187766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:03.188394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:03.188676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:03.188954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:03.255420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:03.257064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:03.284312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1092/2170] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:39:06.432900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:06.433499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:06.433704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:06.454524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:06.455748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:06.512894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1079/2171] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:39:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:06.525509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:06.527598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:06.559803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:06.602561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2b2f-e0ba-48dc-ba45-a2def77b6e7e service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:06.607554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:06.608175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:06.608451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:06.608742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:06.617749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:06.620416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a08458-01b1-447e-b1e0-948d4b50122c service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:06.629627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1093/2172] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:39:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 14:39:07.484015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:07.547737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c9eb2-946c-4642-991e-5f4955cfcffb None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:07.552518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:07.553193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:07.553480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:07.553758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] PATH_INFO: `/v3/users/27c1b20f3ff945bf98e7e49f83bef79c` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:07.554920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c1b20f3ff945bf98e7e49f83bef79c)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:07.557402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:07.659795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:07.738064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c1b20f3ff945bf98e7e49f83bef79c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:07.750866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c1b20f3ff945bf98e7e49f83bef79c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:07.815477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c1b20f3ff945bf98e7e49f83bef79c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:07.828500 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c1b20f3ff945bf98e7e49f83bef79c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:07.829221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c1b20f3ff945bf98e7e49f83bef79c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:07.835238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1080/2173] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:07 2022] DELETE /identity/v3/users/27c1b20f3ff945bf98e7e49f83bef79c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:07.839779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:07.840274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:07.840468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:07.860243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:07.861514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:07.920606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1094/2174] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:07.930398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:07.933017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:07.993333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39513a1f-f2bf-4f43-befe-f0645d76ac65 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:07.997653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:08.001979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:08.001979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:08.001979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:08.012432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:08.014254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:08.033064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1081/2175] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:08.424829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:08.426342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:08.440206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1952-aab3-45a7-8ccf-15ac23e8564e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:08.445019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:08.451740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:08.451740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:08.451740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] PATH_INFO: `/v3/projects/e24142d3aba84f09adf0def2b598df40` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:08.451740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24142d3aba84f09adf0def2b598df40)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:08.451740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:08.575131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24142d3aba84f09adf0def2b598df40', 'request_id': 'req-ec99c201-8fe8-4bc3-bbdd-e982a073182f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.582415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24142d3aba84f09adf0def2b598df40', 'request_id': 'req-ec99c201-8fe8-4bc3-bbdd-e982a073182f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.657717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24142d3aba84f09adf0def2b598df40', 'request_id': 'req-ec99c201-8fe8-4bc3-bbdd-e982a073182f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.669251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] The token cache is being invalidate because project e24142d3aba84f09adf0def2b598df40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:08.669541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.670175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.671703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1095/2176] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:08 2022] DELETE /identity/v3/projects/e24142d3aba84f09adf0def2b598df40 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:08.675660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:08.687997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de2980-3c22-4208-abf3-a8360db31395 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:08.745587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:08.746501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:08.746815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:08.747093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] PATH_INFO: `/v3/users/f344edf6eb094581a01560b793d65941` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:08.748304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f344edf6eb094581a01560b793d65941)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:08.750253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:08.935152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f344edf6eb094581a01560b793d65941'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.937104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:08.947114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f344edf6eb094581a01560b793d65941'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.988681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f344edf6eb094581a01560b793d65941'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.996311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f344edf6eb094581a01560b793d65941'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:08.996947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f344edf6eb094581a01560b793d65941'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.002821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1082/2177] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:08 2022] DELETE /identity/v3/users/f344edf6eb094581a01560b793d65941 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:09.465090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:09.467288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:09.484346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99c201-8fe8-4bc3-bbdd-e982a073182f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:09.528877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:09.547924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:09.548565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:09.548845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:09.549194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] PATH_INFO: `/v3/projects/2b35ae2b19894dad8c39120f0b2af452` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:09.550366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b35ae2b19894dad8c39120f0b2af452)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:09.552258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:09.650332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b35ae2b19894dad8c39120f0b2af452', 'request_id': 'req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.658138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b35ae2b19894dad8c39120f0b2af452', 'request_id': 'req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.751820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b35ae2b19894dad8c39120f0b2af452', 'request_id': 'req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.763305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] The token cache is being invalidate because project 2b35ae2b19894dad8c39120f0b2af452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:09.765513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.765513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:09.765754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1096/2178] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:09 2022] DELETE /identity/v3/projects/2b35ae2b19894dad8c39120f0b2af452 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:09.769720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:09.771231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:09.785764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babedd3a-ce46-4098-b518-bab7b3000f37 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:09.794765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:09.796244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:09.849035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:09.849686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:09.849970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:09.850245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] PATH_INFO: `/v3/users/d7a10efefb714a26880d7f8ac0ebc7f8` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:09.851439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7a10efefb714a26880d7f8ac0ebc7f8)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:09.853408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:09.863683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e5cab-f08c-4236-8324-e4e86b89f9a9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:09.868843 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:09.869845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:09.870310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:09.870642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] PATH_INFO: `/v3/users/227203ab050f43f793d8c4fe3c31497d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:09.871849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=227203ab050f43f793d8c4fe3c31497d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:09.873795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.015680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7a10efefb714a26880d7f8ac0ebc7f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.028770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7a10efefb714a26880d7f8ac0ebc7f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.046037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227203ab050f43f793d8c4fe3c31497d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.058998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227203ab050f43f793d8c4fe3c31497d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.064389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:10.066409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227203ab050f43f793d8c4fe3c31497d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.067257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227203ab050f43f793d8c4fe3c31497d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.114373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7a10efefb714a26880d7f8ac0ebc7f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.123679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7a10efefb714a26880d7f8ac0ebc7f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.124341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7a10efefb714a26880d7f8ac0ebc7f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.130518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1083/2179] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:09 2022] DELETE /identity/v3/users/d7a10efefb714a26880d7f8ac0ebc7f8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:10.165930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227203ab050f43f793d8c4fe3c31497d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.173124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1097/2180] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:09 2022] DELETE /identity/v3/users/227203ab050f43f793d8c4fe3c31497d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:10.428121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:10.429749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.489759 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:10.505783 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29518-2f1e-4251-945a-b8767a238f9a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:10.561921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:10.562488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.562795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.563070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] PATH_INFO: `/v3/projects/a8ee55d2489d40f480f2fc2772712744` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.564222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ee55d2489d40f480f2fc2772712744)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:10.566181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.598185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:10.598185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.611881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0c783-7356-48a6-852d-41f9573ed24e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:10.616485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:10.617160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.617445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.617720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] PATH_INFO: `/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.618894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2892cd8aaa4bc19812dff1d5dbf8f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:10.620808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.675536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ee55d2489d40f480f2fc2772712744', 'request_id': 'req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.730488 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2892cd8aaa4bc19812dff1d5dbf8f1', 'request_id': 'req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.734895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ee55d2489d40f480f2fc2772712744', 'request_id': 'req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.739337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2892cd8aaa4bc19812dff1d5dbf8f1', 'request_id': 'req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.741944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ee55d2489d40f480f2fc2772712744', 'request_id': 'req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.752806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] The token cache is being invalidate because project a8ee55d2489d40f480f2fc2772712744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:10.753125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.753780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3077b5e8-8b1c-42ec-85f9-5d46900c93f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.755361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1084/2181] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:10 2022] DELETE /identity/v3/projects/a8ee55d2489d40f480f2fc2772712744 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:10.760554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612a0a-ff7f-4143-bb80-657955501a44 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.760821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612a0a-ff7f-4143-bb80-657955501a44 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.763214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612a0a-ff7f-4143-bb80-657955501a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.782261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-bf612a0a-ff7f-4143-bb80-657955501a44 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:10.783464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-bf612a0a-ff7f-4143-bb80-657955501a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.841799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1085/2182] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:10.849019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.849314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.849590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.851589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2892cd8aaa4bc19812dff1d5dbf8f1', 'request_id': 'req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.864575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] The token cache is being invalidate because project 9f2892cd8aaa4bc19812dff1d5dbf8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:10.865139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.865799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cf78b2be-6b18-4c9e-9568-e3bff48d4c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:10.867801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1098/2183] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:10 2022] DELETE /identity/v3/projects/9f2892cd8aaa4bc19812dff1d5dbf8f1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:10.871734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:10.875127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe038-a94a-4887-a820-53c507e899a7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.875414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe038-a94a-4887-a820-53c507e899a7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.876392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe038-a94a-4887-a820-53c507e899a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.877653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.886123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1086/2184] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:10.890224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:10.897900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-33dbe038-a94a-4887-a820-53c507e899a7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:10.899060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63421f96-67fc-4fa0-8ba9-26d6042723dd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:10.899175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-33dbe038-a94a-4887-a820-53c507e899a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.903709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:10.904314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.904594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.904865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.905590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:39:10.905893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:10.907851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.907976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1099/2185] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:10.920544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.920911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.921218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.925919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1087/2186] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:39:10.930128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:10.940755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61f52-93dc-46d7-a5c7-621a9314a9e4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:10.942457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:10.945639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:10.945817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:10.946108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.946387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.946662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.947314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:10.950187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.952879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1100/2187] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:10.957092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:10.967491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf69d-9806-4d7b-8d37-65a2b6af5206 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:10.974431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:10.974431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:10.974431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:10.974431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:10.974908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:39:10.974908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:10.976506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:10.991774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1101/2188] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:39:10.996310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.006087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926f94-5234-4fa9-b6b1-5e70c64c1bc9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.011412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.012004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.012283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.012557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.013254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.015133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.048021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1088/2189] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:11.052184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.114561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698f6e-88f0-4c87-888e-0bb6b2fdad0f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.121786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1102/2190] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:10 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:11.122110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.122110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.122110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.122110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.122480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.123340 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.124021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.199233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb7ed-b8d9-46ba-aa02-1c27d9f51321 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.204297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.209378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.209378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.209378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.209378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.209378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.247080 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1089/2191] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:11.248545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.258339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119674e-612e-4da4-950f-782c92268570 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.264201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.264841 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.265152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.265424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.266184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.267690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.281372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1090/2192] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:39:11.282417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.293256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4937d-1113-4344-9a09-6acebdfb902f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.298295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.298796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1103/2193] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:11.300132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.301083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.301083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.301680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.303460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.304298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.318665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1091/2194] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:11.319375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b5f0f-b60b-4938-a12a-1cb8575475cb admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.321581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:11.324239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.325151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.325781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.326064 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.326341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.327050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.328756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.335879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19573d-5a9d-4ba3-82ba-646d826e6740 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.339825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1104/2195] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:39:11.343655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.343790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.344334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.344618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.345068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] PATH_INFO: `/v3/projects/c5c24df7687b4e7093911f73e21b522a/users/3b3047e3ab2046c580e987a6518a4612/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.349108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:11.355859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c7916-e4ae-401a-a38b-fee3142ca4e9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.361165 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.361774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.362097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.362324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.362939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c24df7687b4e7093911f73e21b522a, user_id=3b3047e3ab2046c580e987a6518a4612, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.363056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.364741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.365850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f17a-0052-43c3-88ac-3adf8c960c90 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.377582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1105/2196] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:39:11.386344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.396482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89580f-866d-4a9a-b787-b53b4c6b3a59 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.401237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.401907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.402183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.402459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] PATH_INFO: `/v3/projects/0c18f47fdd154ee9973adbd4fa6614f4/users/782487085da84d38a531fb04665f9fe0/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.417287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c18f47fdd154ee9973adbd4fa6614f4, user_id=782487085da84d38a531fb04665f9fe0, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.419930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865cadc-a386-4dba-b260-623b92983770 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.457890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1092/2197] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:11 2022] PUT /identity/v3/projects/c5c24df7687b4e7093911f73e21b522a/users/3b3047e3ab2046c580e987a6518a4612/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:11.462917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.463236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.463649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.476365 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1106/2198] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:11 2022] PUT /identity/v3/projects/0c18f47fdd154ee9973adbd4fa6614f4/users/782487085da84d38a531fb04665f9fe0/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:11.482931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:11.484145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:11.489853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.490907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.491188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.521736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] MFA Rules not processed for user `782487085da84d38a531fb04665f9fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:11.523808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:11.551206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1093/2199] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:11.561746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.563619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:11.594182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1107/2200] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:11.598493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.603424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:11.611410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07638123-ad67-4e1e-9fa9-c6bacb481688 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.616689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.617358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.617723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.618056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.618802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.620824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.633311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33c3d-e542-40a7-8f8f-ede2613c8dbf None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.638130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.638913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.639220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.639498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.649776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.652068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.662700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1094/2201] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:39:11.727851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1108/2202] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:11.733358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.743586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c896-0534-4235-b241-5b8983a0eded service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.820277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.821095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.821487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.821866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.822803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.824861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.914955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1095/2203] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:11 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:11.919588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.929887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08f0bd-065a-45e2-b600-193a66f841b2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.935810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.936406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.936690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.936966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.937711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.939317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:11.951752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1109/2204] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:11.956329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:11.967773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cae27c-042f-4181-b052-34940e921043 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:11.972823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:11.973526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:11.977361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:11.977361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] PATH_INFO: `/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780/users/c62b1a54dc2f49b6afbf60b238ae560d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:11.991140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d0c4df20c44a61a230f8b0cd7dd780, user_id=c62b1a54dc2f49b6afbf60b238ae560d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:11.993493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:12.064461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1096/2205] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:11 2022] PUT /identity/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780/users/c62b1a54dc2f49b6afbf60b238ae560d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:12.068650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:12.080205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fccc25-00b3-4164-a82b-a3325ddb628b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:12.143684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:12.144327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:12.144677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:12.144962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:12.145699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:12.147307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3bf26-81e2-40c5-817a-7e7bd31d4818 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:12.158251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1110/2206] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:12.162663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:12.173650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564c7e-c36e-4225-ad31-ac839baa6f54 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:12.179272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:12.180009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:12.180297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:12.180879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] PATH_INFO: `/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780/users/c62b1a54dc2f49b6afbf60b238ae560d/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:12.183536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d0c4df20c44a61a230f8b0cd7dd780, user_id=c62b1a54dc2f49b6afbf60b238ae560d, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:12.186534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:12.205410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1097/2207] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:12 2022] PUT /identity/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780/users/c62b1a54dc2f49b6afbf60b238ae560d/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:12.211429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:12.211722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:12.212002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:12.242286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] MFA Rules not processed for user `c62b1a54dc2f49b6afbf60b238ae560d`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:12.244084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:12.259611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:12.331162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1111/2208] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:12.343311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:12.346184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:12.411916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780c59-d650-4c51-bdc6-bdbdb6deabd3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:12.415537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:12.416211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:12.416652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:12.416934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:12.427433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:12.428891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:12.439899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1098/2209] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:12.615241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:12.626468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03487c-ace0-4ed2-80e9-15750633d986 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:12.630247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:12.630865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:12.631146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:12.631421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:12.697040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:12.699300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88324eb-8c1e-4392-ab6a-e20e2d4711bc service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:12.708731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1112/2210] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:13.260680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:13.314531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:13.353323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006c53-7de8-4f76-b9b9-d4b92532c015 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:13.359508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:13.361323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:13.362688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:13.363208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] PATH_INFO: `/v3/users/782487085da84d38a531fb04665f9fe0` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:13.366811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782487085da84d38a531fb04665f9fe0)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:13.371599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:13.611145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782487085da84d38a531fb04665f9fe0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:13.626794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782487085da84d38a531fb04665f9fe0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:13.692129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782487085da84d38a531fb04665f9fe0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:13.702310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782487085da84d38a531fb04665f9fe0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:13.703060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782487085da84d38a531fb04665f9fe0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:13.709471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1099/2211] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:13 2022] DELETE /identity/v3/users/782487085da84d38a531fb04665f9fe0 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:13.714152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:13.714639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:13.714752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:13.734242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:13.735702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:13.800690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1113/2212] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:13.812809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:13.814420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:13.902480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0791-5e2d-4038-b9c9-3b5251e33a91 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:13.909082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:13.909082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:13.909082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:13.909082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:13.921575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:13.923045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:13.934429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1100/2213] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:39:14.567114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:14.568897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:14.582652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20061e-4e71-445a-870d-a15a468ebf73 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:14.593090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:14.593837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:14.594120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:14.594394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] PATH_INFO: `/v3/projects/0c18f47fdd154ee9973adbd4fa6614f4` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:14.596118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c18f47fdd154ee9973adbd4fa6614f4)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:14.598115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:14.735311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:14.741629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c18f47fdd154ee9973adbd4fa6614f4', 'request_id': 'req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:14.747794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c18f47fdd154ee9973adbd4fa6614f4', 'request_id': 'req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:14.812071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c18f47fdd154ee9973adbd4fa6614f4', 'request_id': 'req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:14.823602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] The token cache is being invalidate because project 0c18f47fdd154ee9973adbd4fa6614f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:14.823884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:14.824526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:14.826353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1114/2214] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:14 2022] DELETE /identity/v3/projects/0c18f47fdd154ee9973adbd4fa6614f4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:14.830288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:14.851653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274b683-c1b3-4afa-a288-b9a3b40b5804 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:14.897397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:14.909901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:14.910561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:14.910823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:14.911157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] PATH_INFO: `/v3/users/c62b1a54dc2f49b6afbf60b238ae560d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:14.912326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62b1a54dc2f49b6afbf60b238ae560d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:14.914290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:15.182663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62b1a54dc2f49b6afbf60b238ae560d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:15.199443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62b1a54dc2f49b6afbf60b238ae560d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:15.275439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62b1a54dc2f49b6afbf60b238ae560d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:15.283335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62b1a54dc2f49b6afbf60b238ae560d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:15.284134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-15370182-bf2d-4c23-84e3-71b388c68e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62b1a54dc2f49b6afbf60b238ae560d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:15.290728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1101/2215] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:14 2022] DELETE /identity/v3/users/c62b1a54dc2f49b6afbf60b238ae560d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:15.847976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:15.849490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:15.863676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e71d-b5dd-4c47-95d0-5b29c8a4e1b2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:15.929534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:15.930296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:15.931017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:15.931306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] PATH_INFO: `/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:15.932616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d0c4df20c44a61a230f8b0cd7dd780)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:15.934790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.067195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d0c4df20c44a61a230f8b0cd7dd780', 'request_id': 'req-39b87aa5-a623-4f48-adb1-1f9aa21720b4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:16.075475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d0c4df20c44a61a230f8b0cd7dd780', 'request_id': 'req-39b87aa5-a623-4f48-adb1-1f9aa21720b4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:16.163501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d0c4df20c44a61a230f8b0cd7dd780', 'request_id': 'req-39b87aa5-a623-4f48-adb1-1f9aa21720b4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:16.180913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] The token cache is being invalidate because project 23d0c4df20c44a61a230f8b0cd7dd780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:16.181630 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:16.182586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-39b87aa5-a623-4f48-adb1-1f9aa21720b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:16.184490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1115/2216] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:15 2022] DELETE /identity/v3/projects/23d0c4df20c44a61a230f8b0cd7dd780 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:16.189746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.190226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.191817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.215956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:16.217428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:16.295683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1102/2217] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:16.302686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.303961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.304862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.327456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:16.328825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:16.340454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1116/2218] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:16.344892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.356188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce37e9c-675a-4e45-9a85-460d6c530fd9 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.362331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.362963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.363428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.363711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.364478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:39:16.364793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.366486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.382220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1103/2219] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:39:16.387477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.397400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105e2d-a8a7-48d9-8612-4b84b9303ffa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.404330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.405446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.405801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.406150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.407152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.409891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.498011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1117/2220] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:16.501939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.513256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ee60-3505-4f12-baea-eed5d615cda0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.579033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:16.589862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.590899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.591334 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.591801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.592640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.594713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.736195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1104/2221] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:16 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:16.739987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.751358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4156-e9e8-4204-b24c-1f25bcc24a78 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.757024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.757024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.765206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.765206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.765206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.765206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.775761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.782967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1118/2222] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:16.788756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37263f-5329-4d4a-afb7-abfe20d7387f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.797160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.798616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.798616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.798616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.798616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.800098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee99219-c855-4224-9bdc-6f28e9c88997 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.814956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1105/2223] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:16.819285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:16.831122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451bb9-a901-4199-bb7f-2fda86021333 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:16.836538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:16.837290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.837575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.838327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] PATH_INFO: `/v3/projects/a17dbe88c63c4ec38969891bd1b387cd/users/86a8650ca18d40fea2f67b4591ebe403/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.856251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17dbe88c63c4ec38969891bd1b387cd, user_id=86a8650ca18d40fea2f67b4591ebe403, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:16.859003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:16.861329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:16.956373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1119/2224] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:16 2022] PUT /identity/v3/projects/a17dbe88c63c4ec38969891bd1b387cd/users/86a8650ca18d40fea2f67b4591ebe403/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:16.967043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:16.967336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:16.967687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:16.997537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] MFA Rules not processed for user `86a8650ca18d40fea2f67b4591ebe403`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:16.998770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:17.083802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1106/2225] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:17.089435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.168112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc948e-9d13-4a8c-9f25-322b461e3c71 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.174856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.175755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.176228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.176674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.177424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.179876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.289870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1120/2226] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:17.292094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.309310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568431-5644-4b36-ab10-14e3156dca12 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.415133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.415883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.416791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.417215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.418006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.420208 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.501067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1107/2227] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:17.505477 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.519493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74118f-c10a-4f90-8c5c-a75ee629d0ea admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.528815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.529934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.530263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.530596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.531509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.533253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.553036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1121/2228] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:17.554435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.578059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ec7f-2b07-4284-a324-8db322f60ea6 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.597275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.597860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.598144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.623236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.624076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.631178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82955d8c-7bf2-4887-9cac-d5d95dd670d0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.653068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1108/2229] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:17.653180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.655676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.656004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.656320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.674647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c0520-d19e-4fd0-9761-be54b1bdf559 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.682594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.683542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.693185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.693551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] PATH_INFO: `/v3/projects/6445146a25d14aa1b3bdf62fb6a81e4e/users/b55fa64e44284b4e908ffef503da69ff/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.695770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] MFA Rules not processed for user `3b3047e3ab2046c580e987a6518a4612`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:17.699408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:17.713179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6445146a25d14aa1b3bdf62fb6a81e4e, user_id=b55fa64e44284b4e908ffef503da69ff, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.715987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9963ea-1e27-4422-bee7-dd5369ebc993 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.737979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:17.781730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1109/2230] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:17.786232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.787729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:17.802796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508568b-2b31-4938-99ae-cfdac0e07e4c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:17.808649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:17.809441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.809727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.810004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.810696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:17.812698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:17.832222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1122/2231] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:17 2022] PUT /identity/v3/projects/6445146a25d14aa1b3bdf62fb6a81e4e/users/b55fa64e44284b4e908ffef503da69ff/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:17.907850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:17.908511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:17.908511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:17.915472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1110/2232] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:17.920383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:17.942112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] MFA Rules not processed for user `b55fa64e44284b4e908ffef503da69ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:17.943379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:17.999004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b307d-cf48-481b-8600-ac001bd47516 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.004382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.005167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.005456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.005735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.006407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.008619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.030460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1123/2233] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:18.034985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.047458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872376ff-9515-4743-8d20-d80ca1f41108 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.052375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.053141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.053449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.053797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.054475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.056957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.108477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1111/2234] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:17 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:18.112635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.123172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd4bea-8b6a-42dc-afb9-f0acc139049c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.129115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.133456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.133958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.136777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.138545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.139485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.152269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1112/2235] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:39:18.156884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.167536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af03ae-29e4-4417-95c6-6418cb309bf4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.172543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.173487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.174083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.174389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] PATH_INFO: `/v3/projects/d1dcbd182e61408bae721537b0dcf329/users/234b55dae5f8430f98f3eec3315b2368/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.191743 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dcbd182e61408bae721537b0dcf329, user_id=234b55dae5f8430f98f3eec3315b2368, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.194877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.200269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1124/2236] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:18 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:18.204069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.220337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:18.273133 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab5bc-8f77-4be1-a2d2-2a31f66d85d8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.278357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1113/2237] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:18 2022] PUT /identity/v3/projects/d1dcbd182e61408bae721537b0dcf329/users/234b55dae5f8430f98f3eec3315b2368/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:18.282571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.335884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.336444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.336725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.337016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.337723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.340101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.353020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e8cd6-32af-46ac-b4f3-9e88790d23a3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.358563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.359197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.359482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.359762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.363800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.363800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.365638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:18.375037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1114/2238] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:39:18.379086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.389594 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6794-d3a8-4209-a1a9-dc04da42b292 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.394659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.395362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.395642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.401371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] PATH_INFO: `/v3/projects/d1dcbd182e61408bae721537b0dcf329/users/234b55dae5f8430f98f3eec3315b2368/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.401371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dcbd182e61408bae721537b0dcf329, user_id=234b55dae5f8430f98f3eec3315b2368, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.401371 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.411572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1115/2239] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:18 2022] PUT /identity/v3/projects/d1dcbd182e61408bae721537b0dcf329/users/234b55dae5f8430f98f3eec3315b2368/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:18.427196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1125/2240] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:18 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:18.431469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.500394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de813a-ff81-4bef-bf51-6a02c5dc81a8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.505246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.505893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.506212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.506523 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.507309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.509049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.523033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1116/2241] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:18.527818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.539181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf9a96-30c3-4c30-94d6-45c2015f6434 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.544434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.545349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.545635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.545913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] PATH_INFO: `/v3/projects/b0b504f2693d4085ad263f552a8a60a4/users/abfe16af24b34e83a5ca811da365065b/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.563118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b504f2693d4085ad263f552a8a60a4, user_id=abfe16af24b34e83a5ca811da365065b, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.566325 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.650214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1126/2242] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:18 2022] PUT /identity/v3/projects/b0b504f2693d4085ad263f552a8a60a4/users/abfe16af24b34e83a5ca811da365065b/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:18.651178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.661215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b329e1-0c69-4a64-afcd-194f8a342d5f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.727445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.728085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.728389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.729572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.730512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.732186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99837ae7-0cdb-41ad-aff4-d5bbb0dc0cea admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.747077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1117/2243] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:18.747545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.758979 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6efc-db9f-44a9-ad0b-177110a223f7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.764252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.765203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.765546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.765883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] PATH_INFO: `/v3/projects/b0b504f2693d4085ad263f552a8a60a4/users/abfe16af24b34e83a5ca811da365065b/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.767918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b504f2693d4085ad263f552a8a60a4, user_id=abfe16af24b34e83a5ca811da365065b, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:18.770601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:18.783108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1127/2244] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:18 2022] PUT /identity/v3/projects/b0b504f2693d4085ad263f552a8a60a4/users/abfe16af24b34e83a5ca811da365065b/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:18.793707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.793997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.794314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:18.825691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] MFA Rules not processed for user `abfe16af24b34e83a5ca811da365065b`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:18.826903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:18.899950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1118/2245] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:18.912769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:18.915148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:18.978214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97ac7e-fb9d-40e6-9238-2d4cd949db5c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:18.983131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:18.983781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:18.984061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:18.984334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:19.012927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:19.047137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:19.048574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:19.059439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1128/2246] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:22.373496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:22.379751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:22.386121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83dd5-3cf9-4e93-b18f-5bb478fc2045 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:22.390671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:22.391360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:22.392009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:22.392438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:22.463462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:22.465319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db139e9e-6ca0-4042-8d92-7a3ce3273d98 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:22.477326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1119/2247] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:24.048078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:24.057675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823b5c4-29f8-4e1f-afc4-d6bc0faa76a8 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:24.070275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:24.071122 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:24.071538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:24.072490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:24.073778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3fe09-604a-4bbd-a66c-818a5bd32d01 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:24.074572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3fe09-604a-4bbd-a66c-818a5bd32d01 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:24.075105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3fe09-604a-4bbd-a66c-818a5bd32d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:24.083256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:24.087361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:24.099152 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1129/2248] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:24.112170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8ef3fe09-604a-4bbd-a66c-818a5bd32d01 None None] MFA Rules not processed for user `234b55dae5f8430f98f3eec3315b2368`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:24.113453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8ef3fe09-604a-4bbd-a66c-818a5bd32d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:24.204368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1120/2249] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:39:24.218983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:24.220544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:24.367951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70c4ae-8eb0-41f2-a114-03338bc74691 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:24.379548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:24.379548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:24.379548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:24.379548 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:24.448806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:24.450366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:24.465462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1130/2250] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:24.754656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:24.754656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:24.754656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:24.777544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:24.778758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:24.842166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1121/2251] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:24.855113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:24.856742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:24.918495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:24.941727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d165c-156d-4c38-a91c-55a74cf1bd75 service glance] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:24.947814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:24.948449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:24.948894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:24.949032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:24.958604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:24.960316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded9d44-4ea2-4402-bab0-64cf5044af24 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:24.970662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1131/2252] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:39:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:39:30.057399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:30.097668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:30.141204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec47c8-44e5-4a04-9ab4-791ae32a9e3e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:30.147024 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:30.147950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:30.148397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:30.148825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] PATH_INFO: `/v3/users/86a8650ca18d40fea2f67b4591ebe403` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:30.151150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a8650ca18d40fea2f67b4591ebe403)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:30.153350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:30.355419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a8650ca18d40fea2f67b4591ebe403'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:30.371583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a8650ca18d40fea2f67b4591ebe403'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:30.443795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a8650ca18d40fea2f67b4591ebe403'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:30.453238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a8650ca18d40fea2f67b4591ebe403'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:30.454738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a8650ca18d40fea2f67b4591ebe403'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:30.464985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1122/2253] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:30 2022] DELETE /identity/v3/users/86a8650ca18d40fea2f67b4591ebe403 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:30.469491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:30.475346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:30.475346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:30.495667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:30.496896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:30.570596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1132/2254] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:30.581044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:30.582482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:30.651344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1c9b1-77be-48aa-8080-a1ae4993fefa admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:30.655105 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:30.655839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:30.656133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:30.656408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:30.666991 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:30.668534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:30.680489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1123/2255] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:39:31.485863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:31.487502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:31.500875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d125d5-6763-43ff-873b-7f435c87df68 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:31.509043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:31.509749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:31.510034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:31.510373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] PATH_INFO: `/v3/projects/a17dbe88c63c4ec38969891bd1b387cd` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:31.511728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17dbe88c63c4ec38969891bd1b387cd)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:31.513722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:31.632590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17dbe88c63c4ec38969891bd1b387cd', 'request_id': 'req-e9b796e0-e780-4a01-8661-50cb22b49a89'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:31.639578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17dbe88c63c4ec38969891bd1b387cd', 'request_id': 'req-e9b796e0-e780-4a01-8661-50cb22b49a89'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:31.762883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17dbe88c63c4ec38969891bd1b387cd', 'request_id': 'req-e9b796e0-e780-4a01-8661-50cb22b49a89'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:31.776759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] The token cache is being invalidate because project a17dbe88c63c4ec38969891bd1b387cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:31.777478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:31.778370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:31.783107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1133/2256] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:31 2022] DELETE /identity/v3/projects/a17dbe88c63c4ec38969891bd1b387cd => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:31.787354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:31.801096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a541bd-eeda-4991-8480-294f98054e92 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:31.866993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:31.878631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:31.879286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:31.879571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:31.879922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] PATH_INFO: `/v3/users/b55fa64e44284b4e908ffef503da69ff` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:31.881328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55fa64e44284b4e908ffef503da69ff)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:31.883386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:32.083525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55fa64e44284b4e908ffef503da69ff'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.103895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55fa64e44284b4e908ffef503da69ff'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.210807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55fa64e44284b4e908ffef503da69ff'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.222488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55fa64e44284b4e908ffef503da69ff'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.223239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55fa64e44284b4e908ffef503da69ff'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.230170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1124/2257] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:31 2022] DELETE /identity/v3/users/b55fa64e44284b4e908ffef503da69ff => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:32.708078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:32.708078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:32.723760 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b796e0-e780-4a01-8661-50cb22b49a89 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:32.800739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:32.803990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:32.803990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:32.803990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] PATH_INFO: `/v3/projects/6445146a25d14aa1b3bdf62fb6a81e4e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:32.803990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6445146a25d14aa1b3bdf62fb6a81e4e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:32.806031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:32.808702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:32.935673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6445146a25d14aa1b3bdf62fb6a81e4e', 'request_id': 'req-bd031bab-8c3d-4a41-9bd4-720295a92d47'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:32.942695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6445146a25d14aa1b3bdf62fb6a81e4e', 'request_id': 'req-bd031bab-8c3d-4a41-9bd4-720295a92d47'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.051027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6445146a25d14aa1b3bdf62fb6a81e4e', 'request_id': 'req-bd031bab-8c3d-4a41-9bd4-720295a92d47'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.068270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] The token cache is being invalidate because project 6445146a25d14aa1b3bdf62fb6a81e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:33.068556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.069269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.070852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1134/2258] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:32 2022] DELETE /identity/v3/projects/6445146a25d14aa1b3bdf62fb6a81e4e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:33.075420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:33.076839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:33.090945 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6f70-5c00-40fc-b6a1-b794e64e4438 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:33.170545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:33.171335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:33.171619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:33.171887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] PATH_INFO: `/v3/users/abfe16af24b34e83a5ca811da365065b` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:33.173112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfe16af24b34e83a5ca811da365065b)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:33.175048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:33.376470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfe16af24b34e83a5ca811da365065b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.391466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfe16af24b34e83a5ca811da365065b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.473609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfe16af24b34e83a5ca811da365065b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.476061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:33.482641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfe16af24b34e83a5ca811da365065b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.483354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-955ee6ba-36c7-43ca-9ed4-2fa92ad0f7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfe16af24b34e83a5ca811da365065b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:33.490438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1125/2259] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:39:33 2022] DELETE /identity/v3/users/abfe16af24b34e83a5ca811da365065b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:34.352543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:34.354236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:34.371940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031bab-8c3d-4a41-9bd4-720295a92d47 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:34.433116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:34.433440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:34.433694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:34.434247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] PATH_INFO: `/v3/projects/b0b504f2693d4085ad263f552a8a60a4` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:34.435564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b504f2693d4085ad263f552a8a60a4)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:34.438287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:34.598835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b504f2693d4085ad263f552a8a60a4', 'request_id': 'req-672c3a9b-b434-4799-8256-ae7df6988212'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:34.605350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b504f2693d4085ad263f552a8a60a4', 'request_id': 'req-672c3a9b-b434-4799-8256-ae7df6988212'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:34.725293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b504f2693d4085ad263f552a8a60a4', 'request_id': 'req-672c3a9b-b434-4799-8256-ae7df6988212'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:34.739912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] The token cache is being invalidate because project b0b504f2693d4085ad263f552a8a60a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:39:34.739912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:34.739912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-672c3a9b-b434-4799-8256-ae7df6988212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:39:34.739912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1135/2260] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:39:34 2022] DELETE /identity/v3/projects/b0b504f2693d4085ad263f552a8a60a4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:34.745147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:34.745587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:34.745686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:34.769421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:34.769421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:34.841724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1126/2261] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:34.850920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:34.851148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:34.851433 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:34.876389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:34.877657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:34.887066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1136/2262] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:34.890972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:34.901870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472245b-e3cc-4dec-b25f-e0210f338683 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:34.906452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:34.907068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:34.907348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:34.907621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:34.908335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:39:34.908616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:34.910522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:34.925694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1127/2263] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:39:34.932355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:34.942387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020698-df0f-4ec7-9d34-97eaa498e4d2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:34.950058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:34.950685 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:34.950965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:34.951230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:34.951896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:34.953818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:34.966841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:35.050969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1137/2264] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:34 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:39:35.053818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:35.065710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7eeda-1219-4cda-9a8c-fcb50ebb511b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:35.128561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:35.129230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.129511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.129782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.130441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:35.132408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:35.252014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1128/2265] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:35 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:35.256185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:35.267361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a176639-bc76-4559-8a69-eaa21ce8500c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:35.271879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:35.272479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.272755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.273054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.273741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:35.275848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:35.286014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1138/2266] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:35.290112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:35.300267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44961b90-ad1f-4bd1-8d84-f40c5ad28fd2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:35.305253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:35.306380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.306674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.306987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.307750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:35.309473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0425551-6697-4241-bd34-63d1fe723365 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:35.320771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1129/2267] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:35.325506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:35.335860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293d760-adc4-4e1d-9c9d-41c85e2008e2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:35.340882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:35.343385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.343385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.343385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] PATH_INFO: `/v3/projects/10f1fba5db2b4402b2fe8d12d099e331/users/f9360a76354e4ae99668fe8debe4e7fb/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.356908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f1fba5db2b4402b2fe8d12d099e331, user_id=f9360a76354e4ae99668fe8debe4e7fb, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:35.359588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:35.425410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1139/2268] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:35 2022] PUT /identity/v3/projects/10f1fba5db2b4402b2fe8d12d099e331/users/f9360a76354e4ae99668fe8debe4e7fb/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:35.431396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9daf-bc3d-49e8-b5ad-dcbfc8a06ad2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.431682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9daf-bc3d-49e8-b5ad-dcbfc8a06ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.431959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b9daf-bc3d-49e8-b5ad-dcbfc8a06ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.451522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-698b9daf-bc3d-49e8-b5ad-dcbfc8a06ad2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:35.452713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-698b9daf-bc3d-49e8-b5ad-dcbfc8a06ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:35.506657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:35.517548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1130/2269] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:35.527571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:35.529319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:35.589749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbc7a-32cc-46e4-bc80-62657f8531ad admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:35.593561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:35.594206 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.594490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.594795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.606144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:35.607607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf7b8e-456e-4f28-9526-d895f15128b0 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:35.618057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1140/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:39:35.791207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:35.791786 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:35.791888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:35.811546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] MFA Rules not processed for user `3b3047e3ab2046c580e987a6518a4612`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:35.812755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:35.870299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1131/2271] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:40.431346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d720b-4a0a-496e-8594-4b51def516a1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:40.437952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d720b-4a0a-496e-8594-4b51def516a1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:40.437952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d720b-4a0a-496e-8594-4b51def516a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:40.452775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-988d720b-4a0a-496e-8594-4b51def516a1 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:40.454045 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-988d720b-4a0a-496e-8594-4b51def516a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:40.524189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1141/2272] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:40.537581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:40.539723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:40.661437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61672c07-eebe-4974-92a1-a54bd8a2825d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:40.667775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:40.668505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:40.668864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:40.688053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:40.711295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:40.713961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:40.729385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1132/2273] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:39:42.103879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.104409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.104409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.167222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] MFA Rules not processed for user `f9360a76354e4ae99668fe8debe4e7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:42.174103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:42.247560 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:42.264212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1142/2274] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:42.269480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:42.298323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc110f14-9659-46a6-b891-5e56fde99f96 service swift] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:42.303432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:42.304061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.304348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.304889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.306204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:42.308382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:42.431956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1133/2275] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:39:42 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:42.435743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:42.459769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e10ec1-446d-484d-a394-2fd68cdefcd8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:42.615199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:42.615834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.616112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.616387 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.617109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:42.619126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:42.714933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1143/2276] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:39:42 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:39:42.719154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:42.732747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e102-7c09-448c-93b4-cc191aac484d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:42.737971 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:42.738733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.739087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.739435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.740271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:42.742091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:42.757282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1134/2277] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:42.758492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:42.769698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c4fe-e383-4c03-90d9-140cbbca68cf admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:42.777200 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:42.778129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.778414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.778690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] PATH_INFO: `/v3/projects/85ea9dc0532d4b02a304af748e008b96/users/fe5ed1be57264d96834671e0d31747cc/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.797043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ea9dc0532d4b02a304af748e008b96, user_id=fe5ed1be57264d96834671e0d31747cc, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:42.800058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:42.875933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1144/2278] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:42 2022] PUT /identity/v3/projects/85ea9dc0532d4b02a304af748e008b96/users/fe5ed1be57264d96834671e0d31747cc/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:39:42.880927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:42.892152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53a740-eb01-48ea-b236-7f5b3ddec812 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:42.978709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:42.979202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:42.979484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:42.979788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:42.980492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:42.982156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b6e1f-31ec-4e4b-acad-791f6cae23c3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:42.983880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:42.993940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1135/2279] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:39:42.998223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:43.009343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6181-b4a5-4cb4-a1b5-d8f534d16dc4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:43.014386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:43.015077 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:43.015359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:43.015637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] PATH_INFO: `/v3/projects/85ea9dc0532d4b02a304af748e008b96/users/fe5ed1be57264d96834671e0d31747cc/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:43.017741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ea9dc0532d4b02a304af748e008b96, user_id=fe5ed1be57264d96834671e0d31747cc, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:43.020570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:43.038368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1145/2280] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:39:42 2022] PUT /identity/v3/projects/85ea9dc0532d4b02a304af748e008b96/users/fe5ed1be57264d96834671e0d31747cc/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:39:50.856121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aca4fb-2765-4167-92a0-67c02f1684ff None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:50.856978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aca4fb-2765-4167-92a0-67c02f1684ff None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:50.856978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aca4fb-2765-4167-92a0-67c02f1684ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:50.891235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-38aca4fb-2765-4167-92a0-67c02f1684ff None None] MFA Rules not processed for user `fe5ed1be57264d96834671e0d31747cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:39:50.898886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-38aca4fb-2765-4167-92a0-67c02f1684ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:50.993968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1136/2281] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:39:51.011992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:39:51.013903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:39:51.113341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978ba27-68b3-4807-b11f-82f44b177d2d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:39:51.120231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:39:51.120875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:39:51.121195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:39:51.121475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:39:51.177942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:39:51.193083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:39:51.194592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d6f40-73e3-4e86-b964-35504914cf2b service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:39:51.204515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1146/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:00.990330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=36 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:40:01.244076 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=8 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:40:03.420133 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:03.420682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:03.420795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:03.442802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] MFA Rules not processed for user `30432a068ad8486db8af635e66776f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:03.444042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:03.504719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1137/2283] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:13.497610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=25 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:40:20.521684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ec3d-c18a-43be-8301-e97776c12ca3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:20.522292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ec3d-c18a-43be-8301-e97776c12ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:20.522292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ec3d-c18a-43be-8301-e97776c12ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:20.542198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-14b5ec3d-c18a-43be-8301-e97776c12ca3 None None] MFA Rules not processed for user `f9360a76354e4ae99668fe8debe4e7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:20.542995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-14b5ec3d-c18a-43be-8301-e97776c12ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:20.550870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1147/2284] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:27.559428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:27.561317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:27.627171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c2cb5-ca6a-4501-99fa-4ff9a3c1a038 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:27.643929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:27.644787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:27.645197 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:27.645569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:27.647849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:27.648220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:27.649121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:27.649672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:27.683921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:27.683921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:27.739845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:27.741564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:27.753602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1138/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:40:27.759529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1148/2286] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:40:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:40:27.785716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:27.878929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1f16-07ea-40bf-913d-0126f84f94dd service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:27.885898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:27.886944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:27.887235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:27.887507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:27.900773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:27.903364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:27.912887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1139/2287] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:40:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:40:34.340529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:34.350913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895cbd5-b495-4827-847d-0d5c6ba00316 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:34.354899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:34.356070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:34.356345 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:34.357019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:34.358772 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:34.374604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:34.376375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:34.389086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1149/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:34.598274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:34.608572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2853ab-79f7-4339-acb6-d16cb09a6a5e service swift] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:34.612699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:34.613340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:34.613618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:34.613979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:34.615817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:34.629985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:34.631479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622212f-d377-4c10-84a2-f7ca4874c044 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:34.645276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1140/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:42.739902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:42.748148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=35 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:40:42.754478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc800-94cc-471c-acaf-005edce98ac5 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:42.760219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:42.760834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:42.761174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:42.761574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:42.763638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:42.827663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:42.829218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583b93f-5d0d-497f-9875-ac516dc83de2 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:42.839123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1150/2290] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:44.639545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=88 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:40:46.950252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:46.951266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:46.951717 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:46.955217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1141/2291] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:40:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:40:46.967303 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd348a50-1ee3-4012-92fd-2e020bdcee70 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:46.968188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd348a50-1ee3-4012-92fd-2e020bdcee70 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:46.968624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd348a50-1ee3-4012-92fd-2e020bdcee70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:46.993241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-cd348a50-1ee3-4012-92fd-2e020bdcee70 None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:46.994451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cd348a50-1ee3-4012-92fd-2e020bdcee70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:47.015559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1151/2292] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:47.028163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:47.030241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:47.093461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574577d-0da0-4cf2-963a-b68042ad23d5 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:47.096961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:47.097606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.097891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.098163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.108887 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:47.110369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76350eb9-8b51-4e09-849a-95f6c1bce761 service glance] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:47.121737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1142/2293] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:47.387038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64a95-9f17-48bb-8464-9ca5bb0f18ba None None] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.387243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64a95-9f17-48bb-8464-9ca5bb0f18ba None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.387503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64a95-9f17-48bb-8464-9ca5bb0f18ba None None] PATH_INFO: `/` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.388874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1152/2294] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:40:47.393555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec580-f0bf-4a29-a550-54d2326bf687 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.393842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec580-f0bf-4a29-a550-54d2326bf687 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.394121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec580-f0bf-4a29-a550-54d2326bf687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.414555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b83ec580-f0bf-4a29-a550-54d2326bf687 None None] MFA Rules not processed for user `06cce26ee600472f84a9d8f7b782bdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:47.415797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b83ec580-f0bf-4a29-a550-54d2326bf687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:47.423026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1143/2295] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:47.434915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c64a95-9f17-48bb-8464-9ca5bb0f18ba None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:47.446457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c64a95-9f17-48bb-8464-9ca5bb0f18ba None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:47.450029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:47.450631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.450995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.451256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.458469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:47.459883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e76bc3-be83-4c06-8e6a-e427950dc0ea service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:47.468110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1153/2296] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:47.549534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce574-5729-444d-b73a-1ed823c85476 None None] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.549819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce574-5729-444d-b73a-1ed823c85476 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.550095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce574-5729-444d-b73a-1ed823c85476 None None] PATH_INFO: `/` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.551630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1144/2297] 10.222.0.49 () {54 vars in 921 bytes} [Fri Jul 29 14:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 29 14:40:47.557123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.557414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.557728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.577770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] MFA Rules not processed for user `e4a3cbcc38d741178be408abd4156959`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:47.578969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:47.588571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1154/2298] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Jul 29 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:47.601417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce574-5729-444d-b73a-1ed823c85476 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:47.613137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce574-5729-444d-b73a-1ed823c85476 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:47.616747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:47.617386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:47.617668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:47.617942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:47.628276 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:47.629739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583881ca-26f4-48fb-90a8-eea2b268b440 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:47.640483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1145/2299] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:48.384832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:48.395009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e23d6-069b-473c-bb43-61aa8bfc0d2d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:48.398541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:48.399143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:48.399422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:48.399691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:48.409589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:48.413042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:48.421502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1155/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:40:52.363950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:52.364431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:52.364526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:52.383310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:40:52.384506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:52.392260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1146/2301] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:40:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:40:52.405381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:52.419771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02bdb1-f5dc-4454-a45d-e0846924f93e service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:52.424914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:52.425551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:52.425831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:52.426107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:52.434918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:52.436478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:52.444572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1156/2302] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:40:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:40:58.814633 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:40:58.832554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817bec5-d368-4435-ac3f-75b7790df398 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:40:58.838379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:40:58.839151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:40:58.839465 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:40:58.839909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:40:58.843432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:40:58.915725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:40:58.917939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c8c39-71b8-45be-a16c-e7204b084bf2 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:40:58.929343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1147/2303] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:40:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:41:02.441267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=51 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:41:08.924149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=66 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:41:21.516660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:41:21.526882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad935-be32-49c4-a4fe-80b91fc46f7f service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:41:21.536751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:41:21.537572 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:41:21.537885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:41:21.538162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:41:21.540042 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:41:21.550429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:41:21.552135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:41:21.560597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1157/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:41:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:41:26.558915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529fc0-0b14-410e-8b0c-9c197df46f5b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:41:26.559407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529fc0-0b14-410e-8b0c-9c197df46f5b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:41:26.559506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529fc0-0b14-410e-8b0c-9c197df46f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:41:26.580094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-61529fc0-0b14-410e-8b0c-9c197df46f5b None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:41:26.581339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-61529fc0-0b14-410e-8b0c-9c197df46f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:41:26.588546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1148/2305] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:41:26.602343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:41:26.614929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c67a7-e573-4ed8-98ad-8b134bb0d8f8 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:41:26.619812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:41:26.620420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:41:26.620701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:41:26.620978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:41:26.629771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:41:26.631342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:41:26.639337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1158/2306] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:41:36.573984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:41:36.636124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=12 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:41:48.499540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:41:48.500035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:41:48.500130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:41:48.520226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] MFA Rules not processed for user `f4ae3fb6a7954a24b88df4f35f55020c`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:41:48.521452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:41:48.528259 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1149/2307] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:41:56.019422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:41:56.031957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862f86-d29b-49f6-805a-2165bf28e6b9 service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:41:56.038395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:41:56.041958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:41:56.042311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:41:56.042665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:41:56.044578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:41:56.058224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:41:56.059829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:41:56.070753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1159/2308] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:41:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:41:58.513534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:03.944733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:03.957995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064ebdb-489c-432d-b8bf-2435fa7fff3f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:03.963650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:03.964264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:03.964573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:03.964847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:03.966603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:04.024800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:04.026426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:04.035662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1150/2309] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:06.066168 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:10.850328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:10.861432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1587c4-e0c3-4bf8-ab90-fc3a0c6056bf service nova] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:10.867718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:10.868326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:10.868606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:10.868875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:10.870620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:10.881623 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:10.883182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad00274-808e-4f37-974d-e325ffa27aa6 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:10.892070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1160/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:14.031225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=28 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:14.924256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:14.927647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:14.945702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca875e-fd85-4760-9392-e682c86dc3fa service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:14.951595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:14.954539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:14.954539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:14.954539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:14.957111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:14.977015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:14.977015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:14.986568 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1151/2311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:20.887923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:24.982071 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:27.795716 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:27.796237 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:27.796344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:27.816228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] MFA Rules not processed for user `30432a068ad8486db8af635e66776f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:42:27.817454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:27.824247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1161/2312] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:42:28.127506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:28.136867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10964a3d-37d2-424d-aa65-0847879d70ae service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:28.140604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:28.141248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:28.141532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:28.141809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:28.143554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:28.153645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:28.155083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:28.164664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1152/2313] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:28.551447 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:28.552933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:28.623682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81186a-9c00-4a58-a2c6-c74673474afa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:28.629071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:28.629701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:28.629978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:28.630253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:28.631988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:28.643019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:28.645412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca3069-eaf6-45d9-89ad-eb1850fae622 service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:28.655087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1162/2314] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:38.159847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:38.651026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=35 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:39.585053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:42:39.600343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b5e9-7bdd-4273-9ef7-42b3e1119405 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:42:39.605917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:42:39.606546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:39.606825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:39.607107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:39.608889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:39.623721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:42:39.626760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:42:39.640642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1153/2315] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:42:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:42:48.420308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:42:48.420801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:42:48.420931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:42:48.440558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] MFA Rules not processed for user `de769d721b8b49c6a5d7d391412da122`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:42:48.441805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:42:48.449300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1163/2316] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:42:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:42:49.636192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:42:58.435196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:24.772527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:24.782357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0379a5-36bd-4889-b6da-549c84dbc765 service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:24.785999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:24.786601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:24.786881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:24.787157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:24.788881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:24.854652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:24.856246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:24.866046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1154/2317] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:43:29.670063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:29.671500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:29.695235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dccf9-3a70-4c8b-9369-22c54c919573 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:29.702665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:29.703597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:29.704092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:29.704555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:29.706678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:29.723090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:29.731346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:29.746872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1164/2318] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:43:29.849889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:29.851688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:29.871378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf29a26-7400-430a-a4af-72862f78b60f service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:29.876815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:29.877536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:29.877812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:29.878254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] PATH_INFO: `/v3/users/f4ae3fb6a7954a24b88df4f35f55020c` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:29.879722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ae3fb6a7954a24b88df4f35f55020c)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:29.881884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:30.104637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ae3fb6a7954a24b88df4f35f55020c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.120910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ae3fb6a7954a24b88df4f35f55020c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.189631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ae3fb6a7954a24b88df4f35f55020c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.200775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ae3fb6a7954a24b88df4f35f55020c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.201746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a588e07a-e2a8-4132-ae48-512018eb98c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ae3fb6a7954a24b88df4f35f55020c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.207977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1155/2319] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:29 2022] DELETE /identity/v3/users/f4ae3fb6a7954a24b88df4f35f55020c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:30.456334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:30.467958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0db9-5832-4a95-8470-a9530dc2d5d7 service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:30.526599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:30.527242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:30.527523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:30.527797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] PATH_INFO: `/v3/projects/82cd919860e249acb9dea7620506725b` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:30.528949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82cd919860e249acb9dea7620506725b)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:30.530901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:30.659714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82cd919860e249acb9dea7620506725b', 'request_id': 'req-dbb65731-73fb-4e6f-b0a6-05512963958a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.666983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82cd919860e249acb9dea7620506725b', 'request_id': 'req-dbb65731-73fb-4e6f-b0a6-05512963958a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.737096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82cd919860e249acb9dea7620506725b', 'request_id': 'req-dbb65731-73fb-4e6f-b0a6-05512963958a'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.748576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] The token cache is being invalidate because project 82cd919860e249acb9dea7620506725b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:30.748872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.749515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-dbb65731-73fb-4e6f-b0a6-05512963958a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:30.751178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1165/2320] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:30 2022] DELETE /identity/v3/projects/82cd919860e249acb9dea7620506725b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:43:30.756413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:30.756705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:30.757002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:30.776432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:30.777676 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:30.838461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1156/2321] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:30.847090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:30.847402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:30.847683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:30.869043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:30.870234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:30.878072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1166/2322] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:30.882323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:30.891962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925311e-d589-4c14-b3eb-bae820a4564a None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:30.896939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:30.897617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:30.897909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:30.898213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:30.898914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:43:30.900082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:30.901726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:30.903266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:30.914141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1157/2323] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:43:30.919029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:30.929242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458dbda-c551-4413-853d-33f1f10665d7 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:30.933977 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:30.934725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:30.935001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:30.935277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:30.935964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:30.937883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.049866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1167/2324] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:43:30 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:31.054226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.064380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c098aa-dbfd-4cd5-aede-fe4499d69f49 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.118839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.119709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.119709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.120082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.120586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.122515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.208757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1158/2325] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:43:31 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:31.214015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.231189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fab58-ff45-47cd-bf16-09674b63c080 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.236092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.236931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.237310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.237641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.238339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.240080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.250792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1168/2326] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:31.254728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.265650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ec93-b39c-4c45-a33c-1b4eb7e68257 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.270424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.271042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.271323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.271993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.272282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.273993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ef8b4-41ab-4a27-a88d-f96e3783eb86 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.287122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1159/2327] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:31.291393 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.303310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa40b33-6069-4919-907a-08c9291a1bae admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.310078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.310887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.311174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.311435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] PATH_INFO: `/v3/projects/24e7c2af0da540bc99ddefab2643d2ea/users/f4cb38b3075148fcb7cf73e8657f3440/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.331413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e7c2af0da540bc99ddefab2643d2ea, user_id=f4cb38b3075148fcb7cf73e8657f3440, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.338703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.391815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1169/2328] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:31 2022] PUT /identity/v3/projects/24e7c2af0da540bc99ddefab2643d2ea/users/f4cb38b3075148fcb7cf73e8657f3440/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:31.402845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.403139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.403429 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.432459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] MFA Rules not processed for user `f4cb38b3075148fcb7cf73e8657f3440`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:31.433661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:31.501852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1160/2329] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:31.506590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.528277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:31.576604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686a4d4-b9eb-4076-950a-f5b91db72f90 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.584463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.585138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.585427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.585838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.586467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.589149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.694202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1170/2330] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:43:31 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:31.699199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.710228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1c0db-7b5d-437b-b04d-152c83a8cf56 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.771179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.771813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.772099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.772398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.773104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.775621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.780330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:31.895896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1161/2331] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:43:31 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:31.900696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:31.955268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fc164-6c9a-4345-b0b5-80820484f36b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:31.961317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:31.961317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:31.961317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:31.961317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:31.961897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:31.963234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:31.973694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1171/2332] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:31.979233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:32.031941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637adc-d3aa-4ac8-b5db-543b9da8d290 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:32.036499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:32.037185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.037516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.037740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] PATH_INFO: `/v3/projects/f681f0a4a5fe4db091475bc28db1f018/users/f762a06c83214b678c92f74af1cfdbed/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.052920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f681f0a4a5fe4db091475bc28db1f018, user_id=f762a06c83214b678c92f74af1cfdbed, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:32.056812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:32.134964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1162/2333] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:31 2022] PUT /identity/v3/projects/f681f0a4a5fe4db091475bc28db1f018/users/f762a06c83214b678c92f74af1cfdbed/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:32.139124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:32.149360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b672b9-368b-41d6-b155-562ad4fff5aa admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:32.208204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:32.208757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.209058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.209344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.210037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:32.211751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d06c-93b2-48ab-b47e-419511e31e28 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:32.222036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1172/2334] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:32.229362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:32.240181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea82c4-0c5a-43f2-a878-8432473f839a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:32.245550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:32.246209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.248821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.248821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] PATH_INFO: `/v3/projects/f681f0a4a5fe4db091475bc28db1f018/users/f762a06c83214b678c92f74af1cfdbed/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.248821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f681f0a4a5fe4db091475bc28db1f018, user_id=f762a06c83214b678c92f74af1cfdbed, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:32.251499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:32.262569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1163/2335] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:32 2022] PUT /identity/v3/projects/f681f0a4a5fe4db091475bc28db1f018/users/f762a06c83214b678c92f74af1cfdbed/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:43:32.272886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.273208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.273486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.302029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] MFA Rules not processed for user `f762a06c83214b678c92f74af1cfdbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:32.303285 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:32.336496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:32.372514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1173/2336] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:43:32.384064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:32.385844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:32.460444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0800d5a-31e4-46c8-a854-4dc34d9ae4bc admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:32.464777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:32.465419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.465695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.465966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.474967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:32.476430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b1ab-4289-4fa2-8dd2-a28cc83b710e service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:32.486671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1164/2337] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:43:32.690019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:32.700155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08bc92-60d7-4721-aefc-5e1f33c4ba76 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:32.703913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:32.704525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:32.704800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:32.705112 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:32.762985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:32.764435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c0fa9-e97f-462c-b587-812281756e1d service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:32.774785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1174/2338] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:43:38.825704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:38.826209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:38.826319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:38.844697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] MFA Rules not processed for user `f9360a76354e4ae99668fe8debe4e7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:38.845877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:38.887531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:38.902561 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:38.903028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:38.903139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:38.906282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1165/2339] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:38.935037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:38.936339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:39.001751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1175/2340] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:43:39.178758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:39.180728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:39.231014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:39.232430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:39.252528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee1887-804b-481b-923d-8eb23ff92e7f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:39.257396 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:39.258011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:39.258433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:39.259808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:39.267951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:39.269610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:39.278268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1166/2341] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:43:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:43:39.296911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c09f-ad46-4747-a71b-9d29c04cd613 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:39.300533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:39.301180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:39.301527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:39.301722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:39.303515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:39.314537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:39.315975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:39.321882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:39.326735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1176/2342] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:43:44.249445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:44.251295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:44.354313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7547c-7bad-4c3c-ba16-45a5201ed9f5 service swift] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:44.359335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:44.359952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:44.360237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:44.360521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:44.362367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:44.374669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:44.376271 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2938317-2ce8-463d-b480-3e1233485192 service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:44.386790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1167/2343] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:43:47.231573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:47.299136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023da79-9c1a-45e2-8e14-e8449e220be0 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:47.303727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:47.304360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:47.304638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:47.304916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] PATH_INFO: `/v3/users/f4cb38b3075148fcb7cf73e8657f3440` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:47.306222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4cb38b3075148fcb7cf73e8657f3440)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:47.309466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:47.544139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4cb38b3075148fcb7cf73e8657f3440'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:47.559699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4cb38b3075148fcb7cf73e8657f3440'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:47.567865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4cb38b3075148fcb7cf73e8657f3440'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:47.568876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4cb38b3075148fcb7cf73e8657f3440'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:47.679836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4cb38b3075148fcb7cf73e8657f3440'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:47.686892 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1177/2344] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:47 2022] DELETE /identity/v3/users/f4cb38b3075148fcb7cf73e8657f3440 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:47.691402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:47.691920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:47.692351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:47.722220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:47.723585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:47.770480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:47.800091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1168/2345] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:47.813837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:47.815339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:47.891156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb764f6-c904-4fbd-9c99-958a9b86228f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:47.894963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:47.895631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:47.895909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:47.896179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:47.906800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:47.908574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:47.925245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1178/2346] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:43:48.757558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:48.760082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:48.772808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53d7d-5b80-4ac1-87f4-dcdb655b5fc1 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:48.778357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:48.779252 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:48.779536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:48.779813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] PATH_INFO: `/v3/projects/24e7c2af0da540bc99ddefab2643d2ea` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:48.781020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e7c2af0da540bc99ddefab2643d2ea)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:48.783099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:48.929462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e7c2af0da540bc99ddefab2643d2ea', 'request_id': 'req-57a673f5-4375-4ccc-964c-d23e458c580e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.009372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e7c2af0da540bc99ddefab2643d2ea', 'request_id': 'req-57a673f5-4375-4ccc-964c-d23e458c580e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.017612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e7c2af0da540bc99ddefab2643d2ea', 'request_id': 'req-57a673f5-4375-4ccc-964c-d23e458c580e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.052951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] The token cache is being invalidate because project 24e7c2af0da540bc99ddefab2643d2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:49.053181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.054296 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.069367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1169/2347] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:48 2022] DELETE /identity/v3/projects/24e7c2af0da540bc99ddefab2643d2ea => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:49.074304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:49.104877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a689085-f1a6-49de-b06f-8e92bf818b72 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:49.136093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:49.184756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:49.185650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:49.185939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:49.186218 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] PATH_INFO: `/v3/users/f762a06c83214b678c92f74af1cfdbed` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:49.187396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f762a06c83214b678c92f74af1cfdbed)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:49.189559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:49.405084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f762a06c83214b678c92f74af1cfdbed'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.420092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f762a06c83214b678c92f74af1cfdbed'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.431848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f762a06c83214b678c92f74af1cfdbed'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.432624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f762a06c83214b678c92f74af1cfdbed'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.509573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-6be238a3-d2bc-48ca-bf40-f83fcb56a3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f762a06c83214b678c92f74af1cfdbed'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:49.519706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1179/2348] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:49 2022] DELETE /identity/v3/users/f762a06c83214b678c92f74af1cfdbed => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:50.032936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:50.034714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:50.063848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a673f5-4375-4ccc-964c-d23e458c580e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:50.159341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:50.160556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.161818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.162306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] PATH_INFO: `/v3/projects/f681f0a4a5fe4db091475bc28db1f018` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.165263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=f681f0a4a5fe4db091475bc28db1f018)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:50.180590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:50.330585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f681f0a4a5fe4db091475bc28db1f018', 'request_id': 'req-93518190-cf1e-414a-af5b-528eaa97fdde'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:50.391589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f681f0a4a5fe4db091475bc28db1f018', 'request_id': 'req-93518190-cf1e-414a-af5b-528eaa97fdde'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:50.403709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f681f0a4a5fe4db091475bc28db1f018', 'request_id': 'req-93518190-cf1e-414a-af5b-528eaa97fdde'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:50.409442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:50.422713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] The token cache is being invalidate because project f681f0a4a5fe4db091475bc28db1f018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:50.423191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:50.424452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-93518190-cf1e-414a-af5b-528eaa97fdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:50.426486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1170/2349] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:50 2022] DELETE /identity/v3/projects/f681f0a4a5fe4db091475bc28db1f018 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:50.431624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.432120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.432203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.495737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:50.497154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:50.575597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1180/2350] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:50.585358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.585645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.585918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.611528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:50.613003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:50.639174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1171/2351] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:50.643418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:50.655737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c561959-39ab-4023-ac89-3910e7a4e093 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:50.661116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:50.661937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.662378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.662806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.663811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:43:50.665771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:50.666808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:50.681012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1181/2352] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:43:50.685285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:50.695161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef43c6-810c-4ed0-b9d7-f87a4c63a8e0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:50.706444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:50.707147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.707423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.707696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.708365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:50.710570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:50.809448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1172/2353] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:43:50 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:50.813780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:50.820443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:50.824664 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993930a-21eb-4724-ad32-11e75593e16a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:50.887676 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:50.888464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:50.888665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:50.889564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:50.891268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:50.893258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:50.972730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1182/2354] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:43:50 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:43:50.977489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:50.988251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310071a-8b6d-4109-bf3b-cf5608492bc0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.001448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.003131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.004073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.004835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.005659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.009021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.025097 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1173/2355] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:51.026526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.041822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c7ddb-e970-4cad-a08e-addafd2ec24a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.047379 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.048020 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.048570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.048850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.049600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.051513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.063888 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1183/2356] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:43:51.068228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.078473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cc923-318d-491c-bbf8-3ea0800a40c4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.083685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.084385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.084671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.084948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] PATH_INFO: `/v3/projects/0d3e8da4358344eb8e8c8c09c64484df/users/de6065c959e848e8b4e681ce96554666/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.105156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3e8da4358344eb8e8c8c09c64484df, user_id=de6065c959e848e8b4e681ce96554666, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.107891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3fc63-77b0-4428-aaee-ea931f23b7ed admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.208331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.210649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:51.232006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1174/2357] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:51 2022] PUT /identity/v3/projects/0d3e8da4358344eb8e8c8c09c64484df/users/de6065c959e848e8b4e681ce96554666/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:51.232255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24deb7d-de18-4019-bbd0-d89941eed920 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.240008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.240353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.240693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.279746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] MFA Rules not processed for user `de6065c959e848e8b4e681ce96554666`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:51.281119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:51.306504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.307298 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.307579 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.307857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] PATH_INFO: `/v3/users/3b3047e3ab2046c580e987a6518a4612` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.309166 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3047e3ab2046c580e987a6518a4612)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.311131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.353821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1175/2358] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:51.358231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.368635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95935d-c9eb-4fc4-8237-884611e89863 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.373230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.373827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.374131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.374402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.375049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.376931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.470534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1176/2359] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:43:51 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:43:51.474506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.501422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3047e3ab2046c580e987a6518a4612'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:51.513830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3047e3ab2046c580e987a6518a4612'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:51.520958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3047e3ab2046c580e987a6518a4612'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:51.521726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3047e3ab2046c580e987a6518a4612'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:51.536186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53f7e-02f2-4349-bcb5-f49f4fdc7236 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.541251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.543241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.543526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.543808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.545164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.547389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.625400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3047e3ab2046c580e987a6518a4612'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:51.632036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1184/2360] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:51 2022] DELETE /identity/v3/users/3b3047e3ab2046c580e987a6518a4612 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:43:51.656070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1177/2361] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:43:51 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:43:51.660148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.661798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:51.731014 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c404bd-1ccf-445a-9e36-b501c41f548b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.735626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.736244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.736524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.736799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.737532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.739117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:51.753038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1185/2362] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:43:51.754420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:51.765418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4965e1-1775-4001-af22-aa16f7e09d17 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:51.770061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:51.770726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:51.771043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:51.771392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] PATH_INFO: `/v3/projects/a82c2f139d48422f9b1e7ce8387880fd/users/9bb657edaf4541a6afeb8a8acdb4ca2c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:51.786970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82c2f139d48422f9b1e7ce8387880fd, user_id=9bb657edaf4541a6afeb8a8acdb4ca2c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:51.790776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.001553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1178/2363] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:51 2022] PUT /identity/v3/projects/a82c2f139d48422f9b1e7ce8387880fd/users/9bb657edaf4541a6afeb8a8acdb4ca2c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:52.006361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.016908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438ac7-8914-4fce-a34b-2976f478f492 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.075915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.076657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.076811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.077126 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.077829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.079521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.081632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.083869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.093163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1186/2364] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:43:52.094487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.097976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c4729-cf33-44d7-a080-a35fa91fe14e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.104503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.105555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.105555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.105788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f28d8c-42ca-4630-b16c-4dc1f64455ee admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.106066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] PATH_INFO: `/v3/projects/c5c24df7687b4e7093911f73e21b522a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.107503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c24df7687b4e7093911f73e21b522a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.113497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.113639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.113639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.113639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.113639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] PATH_INFO: `/v3/projects/a82c2f139d48422f9b1e7ce8387880fd/users/9bb657edaf4541a6afeb8a8acdb4ca2c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.114697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82c2f139d48422f9b1e7ce8387880fd, user_id=9bb657edaf4541a6afeb8a8acdb4ca2c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.118499 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916f199-2eb0-44d3-8f3f-671fb427cb2f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.132010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1187/2365] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:43:52 2022] PUT /identity/v3/projects/a82c2f139d48422f9b1e7ce8387880fd/users/9bb657edaf4541a6afeb8a8acdb4ca2c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:43:52.144070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.144359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.144636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.184523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] MFA Rules not processed for user `9bb657edaf4541a6afeb8a8acdb4ca2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:52.185799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.262999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1188/2366] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:43:52.276245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.277848 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.290084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c24df7687b4e7093911f73e21b522a', 'request_id': 'req-23a7a57a-ea11-41c9-af2f-44ba64885e55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.345117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da96d3-34f5-4363-b261-d939086773f9 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.413671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.414422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.414828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.415149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.454148 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c24df7687b4e7093911f73e21b522a', 'request_id': 'req-23a7a57a-ea11-41c9-af2f-44ba64885e55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.462610 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c24df7687b4e7093911f73e21b522a', 'request_id': 'req-23a7a57a-ea11-41c9-af2f-44ba64885e55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.473748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] The token cache is being invalidate because project c5c24df7687b4e7093911f73e21b522a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:52.474316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.474957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.476771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1179/2367] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:52 2022] DELETE /identity/v3/projects/c5c24df7687b4e7093911f73e21b522a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:43:52.480983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.481963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.482433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.483435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.505207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1189/2368] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:43:52.549689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a57a-ea11-41c9-af2f-44ba64885e55 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.554426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.555410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.555410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.555687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] PATH_INFO: `/v3/users/234b55dae5f8430f98f3eec3315b2368` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.556904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234b55dae5f8430f98f3eec3315b2368)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.558892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.607210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:52.608762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.621026 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94bd28-059f-406f-9b85-25b4ec27402b service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:52.625102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:52.625723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:52.626011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:52.626290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:52.628251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:52.688945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:52.690417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:52.700109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1190/2369] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:43:52.768039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234b55dae5f8430f98f3eec3315b2368'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.784502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234b55dae5f8430f98f3eec3315b2368'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.837782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234b55dae5f8430f98f3eec3315b2368'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.847039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234b55dae5f8430f98f3eec3315b2368'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.848871 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-58a30a2b-1cd1-4360-a2a9-ce9065041448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234b55dae5f8430f98f3eec3315b2368'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:52.854876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1180/2370] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:52 2022] DELETE /identity/v3/users/234b55dae5f8430f98f3eec3315b2368 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:52.869143 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:53.124765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:53.137034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ce4e5-cfdf-4b99-9c94-19e07e7b86a6 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:53.203153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:53.203813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:53.204169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:53.204445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] PATH_INFO: `/v3/projects/d1dcbd182e61408bae721537b0dcf329` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:53.205613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1dcbd182e61408bae721537b0dcf329)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:53.207562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:53.345989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1dcbd182e61408bae721537b0dcf329', 'request_id': 'req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:53.352661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1dcbd182e61408bae721537b0dcf329', 'request_id': 'req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:53.445805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b3d8-f149-4667-af24-748d6bf3e4be None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:53.446270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b3d8-f149-4667-af24-748d6bf3e4be None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:53.446382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b3d8-f149-4667-af24-748d6bf3e4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:53.459725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1dcbd182e61408bae721537b0dcf329', 'request_id': 'req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:53.468272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c498b3d8-f149-4667-af24-748d6bf3e4be None None] MFA Rules not processed for user `f3a38dc21bc34164aea48065da64823f`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:53.469543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c498b3d8-f149-4667-af24-748d6bf3e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:53.473478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] The token cache is being invalidate because project d1dcbd182e61408bae721537b0dcf329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:53.473794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:53.474445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:53.489300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1191/2371] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:53 2022] DELETE /identity/v3/projects/d1dcbd182e61408bae721537b0dcf329 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:53.537300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1181/2372] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:43:53.568619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:54.597272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:55.109424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:55.911223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:57.676437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:57.678079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:57.749348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6afd3e-ce3e-4594-9efb-18beee2ba1a4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:57.755516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:57.756182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:57.756462 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:57.756734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] PATH_INFO: `/v3/users/de6065c959e848e8b4e681ce96554666` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:57.757925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6065c959e848e8b4e681ce96554666)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:57.760197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:57.945191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:57.948457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6065c959e848e8b4e681ce96554666'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:57.960113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6065c959e848e8b4e681ce96554666'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:57.967651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6065c959e848e8b4e681ce96554666'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:57.969937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6065c959e848e8b4e681ce96554666'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:58.048684 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6065c959e848e8b4e681ce96554666'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:58.055903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1192/2373] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:57 2022] DELETE /identity/v3/users/de6065c959e848e8b4e681ce96554666 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:58.060387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:58.060876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:58.061049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:58.081289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:43:58.081893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:58.150420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1182/2374] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:43:58.160313 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:58.162674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:58.253877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef989343-3fdc-4522-b5a8-eff434ed3a7e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:58.258272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:58.259123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:58.259610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:58.260053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:58.270398 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:58.272363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:58.283415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1193/2375] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:43:58.891199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:58.892656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:43:58.904472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b7068-06f4-4c3e-b919-1bf63e7b09b0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:58.909102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:58.909735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:58.910013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:58.910397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] PATH_INFO: `/v3/projects/0d3e8da4358344eb8e8c8c09c64484df` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:58.911964 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3e8da4358344eb8e8c8c09c64484df)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:58.913444 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:59.027715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3e8da4358344eb8e8c8c09c64484df', 'request_id': 'req-85d59cf6-9525-42e1-92b3-582c4312b98b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.111058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3e8da4358344eb8e8c8c09c64484df', 'request_id': 'req-85d59cf6-9525-42e1-92b3-582c4312b98b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.118795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3e8da4358344eb8e8c8c09c64484df', 'request_id': 'req-85d59cf6-9525-42e1-92b3-582c4312b98b'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.122912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:59.130116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] The token cache is being invalidate because project 0d3e8da4358344eb8e8c8c09c64484df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:43:59.130403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.131050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.132641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1183/2376] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:43:58 2022] DELETE /identity/v3/projects/0d3e8da4358344eb8e8c8c09c64484df => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:43:59.136925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:43:59.148137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e4c6-ce11-4dc8-91c7-8effbeb8da28 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:43:59.218043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:43:59.218784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:43:59.219066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:43:59.219396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] PATH_INFO: `/v3/users/9bb657edaf4541a6afeb8a8acdb4ca2c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:43:59.220594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb657edaf4541a6afeb8a8acdb4ca2c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:43:59.222679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:43:59.420151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:43:59.421480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb657edaf4541a6afeb8a8acdb4ca2c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.434547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb657edaf4541a6afeb8a8acdb4ca2c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.441458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb657edaf4541a6afeb8a8acdb4ca2c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.442219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb657edaf4541a6afeb8a8acdb4ca2c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.523067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa6d1067-b4ae-4d37-860e-7bfbf4684048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb657edaf4541a6afeb8a8acdb4ca2c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:43:59.533221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1194/2377] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:43:59 2022] DELETE /identity/v3/users/9bb657edaf4541a6afeb8a8acdb4ca2c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:00.089454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.089454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:00.100042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d59cf6-9525-42e1-92b3-582c4312b98b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.189908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.190606 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.190959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.191246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] PATH_INFO: `/v3/projects/a82c2f139d48422f9b1e7ce8387880fd` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.192565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82c2f139d48422f9b1e7ce8387880fd)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.194570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.311677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82c2f139d48422f9b1e7ce8387880fd', 'request_id': 'req-4e20a5de-e8c2-42f2-92e5-068c206e0b01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:00.388545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82c2f139d48422f9b1e7ce8387880fd', 'request_id': 'req-4e20a5de-e8c2-42f2-92e5-068c206e0b01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:00.396115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82c2f139d48422f9b1e7ce8387880fd', 'request_id': 'req-4e20a5de-e8c2-42f2-92e5-068c206e0b01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:00.409443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] The token cache is being invalidate because project a82c2f139d48422f9b1e7ce8387880fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:00.409737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:00.410399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-4e20a5de-e8c2-42f2-92e5-068c206e0b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:00.411965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1184/2378] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:00 2022] DELETE /identity/v3/projects/a82c2f139d48422f9b1e7ce8387880fd => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:00.417403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.417916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.418016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.437855 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:00.439106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:00.503782 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1195/2379] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:00.512228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.512525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.512810 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.534913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:00.536274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:00.544457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1185/2380] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:44:00.549293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.560213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07142d-6810-4d73-8c03-3cb22d09e904 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.565351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.565984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.566261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.566534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.567240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:44:00.567543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.569264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.587163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1196/2381] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:44:00.588567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.598562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452b928-cad1-4920-9d7a-3d8a0e413116 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.603440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.604065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.604340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.604626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.605306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.607267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.699468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1186/2382] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:00 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:00.703745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.717065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a90cd-cb14-4737-91c0-6d230b805203 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.779970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.780693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.781081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.781432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.782214 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.784192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.903655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1197/2383] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:00 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:44:00.907655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.918514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae719-fc55-4361-8de6-46c46f62600d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.923029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.923894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.925363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.929287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.929287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.929287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.938526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1187/2384] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:00.942705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.952605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65815-2119-4280-81c9-2246fcd4c694 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.957210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.957812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.958095 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.958390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:00.959062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:00.960643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224a868-04e7-4e06-bfff-b2cea6fc8ec6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:00.971736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1198/2385] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:00.975830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:00.986704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd731e-c342-42d6-aef3-bcf92e9b16f8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:00.991547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:00.992203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:00.992586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:00.992867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] PATH_INFO: `/v3/projects/9709b0303f094823b60ad8af3d3b114f/users/80b1157cf96243888cb340791fb0d424/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.007546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9709b0303f094823b60ad8af3d3b114f, user_id=80b1157cf96243888cb340791fb0d424, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.011243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.082201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1188/2386] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:00 2022] PUT /identity/v3/projects/9709b0303f094823b60ad8af3d3b114f/users/80b1157cf96243888cb340791fb0d424/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:01.093315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3685c29-007d-44df-afba-810068d45e4f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.093606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3685c29-007d-44df-afba-810068d45e4f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.093886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3685c29-007d-44df-afba-810068d45e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.122480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b3685c29-007d-44df-afba-810068d45e4f None None] MFA Rules not processed for user `80b1157cf96243888cb340791fb0d424`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:01.123639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b3685c29-007d-44df-afba-810068d45e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:01.141183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:01.191031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1199/2387] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:44:01.196239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.217600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:01.257620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352131b-4353-499f-a3ef-ce753a52d2bd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.263472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.264124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.264597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.265142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.266211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.268202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.341373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1189/2388] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:01 2022] POST /identity/v3/projects => generated 391 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:01.345422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3685c29-007d-44df-afba-810068d45e4f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.356279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3685c29-007d-44df-afba-810068d45e4f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.415194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.415829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.416118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.416400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.417141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.419127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.499420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1200/2389] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:01 2022] POST /identity/v3/users => generated 302 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:01.503344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.514390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa093d-2e05-4614-bf66-6dc17ffadecc admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.519514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.519672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.519909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.520186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.520886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.522514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.533884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1190/2390] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:01.537982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.547952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73175615-a5a1-41fc-9e44-65727a1990d7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.559022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.559710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.559996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.560277 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] PATH_INFO: `/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da/users/5286c3dbfb9d4aa29221094af2eca65c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.575956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eafd3cd20b044298416b2b0ac8fb0da, user_id=5286c3dbfb9d4aa29221094af2eca65c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.578658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.680605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1201/2391] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:01 2022] PUT /identity/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da/users/5286c3dbfb9d4aa29221094af2eca65c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:01.684547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.694628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e92268-78a1-4422-b72e-e67ff9c8bae0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.753802 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.754425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.754713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.754993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.755690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.757376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66264293-9bbf-4b1d-91fe-40dff4fea3fb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.767943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1191/2392] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:01.772776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.783578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044a666-0c3c-45b3-9310-98a42a1b6d1e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:01.788116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:01.788766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.789085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.789487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] PATH_INFO: `/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da/users/5286c3dbfb9d4aa29221094af2eca65c/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.791430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eafd3cd20b044298416b2b0ac8fb0da, user_id=5286c3dbfb9d4aa29221094af2eca65c, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:01.794127 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:01.804283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1202/2393] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:01 2022] PUT /identity/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da/users/5286c3dbfb9d4aa29221094af2eca65c/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:44:01.815911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:01.816202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:01.816482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:01.844447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] MFA Rules not processed for user `5286c3dbfb9d4aa29221094af2eca65c`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:01.845711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:01.917636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1192/2394] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:01.928916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:01.935542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:02.023265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed3b28-8ff3-4e3e-a1bd-d8af55ccf56e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:02.027754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:02.028372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:02.028668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:02.028926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:02.040647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:02.044705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:02.046184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:02.056647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1203/2395] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:02.103076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:02.104480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:02.116617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42a92-efbf-4f05-9184-6c123de54ac3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:02.121098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:02.121719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:02.122004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:02.122283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:02.134956 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:02.181369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:02.182856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cd50e-07e4-40e9-8821-15eb82670525 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:02.192093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1193/2396] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:07.932469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:07.997176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf49fe-ab32-4a75-9341-afe481110773 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:08.001721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:08.002363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:08.002643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:08.002920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] PATH_INFO: `/v3/users/80b1157cf96243888cb340791fb0d424` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:08.004079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b1157cf96243888cb340791fb0d424)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:08.006043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:08.261601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b1157cf96243888cb340791fb0d424'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:08.275160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b1157cf96243888cb340791fb0d424'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:08.282296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b1157cf96243888cb340791fb0d424'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:08.283264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b1157cf96243888cb340791fb0d424'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:08.336429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b1157cf96243888cb340791fb0d424'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:08.345366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1204/2397] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:07 2022] DELETE /identity/v3/users/80b1157cf96243888cb340791fb0d424 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:08.350652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:08.351265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:08.351687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:08.374255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:08.376816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:08.455913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1194/2398] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:08.469860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:08.471231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:08.534983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246582c8-a8ca-4fec-bca8-8a60cf8547cf admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:08.539116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:08.540809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:08.541124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:08.541405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:08.554069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:08.555537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:08.568317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1205/2399] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:09.105166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:09.106906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:09.143519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476afd-041f-4a98-a852-c46c80320f98 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:09.153073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:09.153073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:09.153073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:09.153073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] PATH_INFO: `/v3/projects/9709b0303f094823b60ad8af3d3b114f` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:09.170564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9709b0303f094823b60ad8af3d3b114f)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:09.172204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:09.309942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9709b0303f094823b60ad8af3d3b114f', 'request_id': 'req-7bf3f862-651a-4c5a-8efa-a90e66e8c174'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.379564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9709b0303f094823b60ad8af3d3b114f', 'request_id': 'req-7bf3f862-651a-4c5a-8efa-a90e66e8c174'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.389304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9709b0303f094823b60ad8af3d3b114f', 'request_id': 'req-7bf3f862-651a-4c5a-8efa-a90e66e8c174'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.406174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] The token cache is being invalidate because project 9709b0303f094823b60ad8af3d3b114f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:09.406512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.407369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.409351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1195/2400] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:09 2022] DELETE /identity/v3/projects/9709b0303f094823b60ad8af3d3b114f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:09.413293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:09.424838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260b969-c749-4a3d-916b-56a0efe62f58 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:09.439860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:09.499986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:09.500815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:09.501267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:09.501792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] PATH_INFO: `/v3/users/5286c3dbfb9d4aa29221094af2eca65c` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:09.503164 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5286c3dbfb9d4aa29221094af2eca65c)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:09.505924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:09.720735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5286c3dbfb9d4aa29221094af2eca65c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.741151 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5286c3dbfb9d4aa29221094af2eca65c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.750625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5286c3dbfb9d4aa29221094af2eca65c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.751516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5286c3dbfb9d4aa29221094af2eca65c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.832370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab902511-c94e-4db1-a86f-e3d1a1b6dc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5286c3dbfb9d4aa29221094af2eca65c'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:09.851899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1206/2401] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:09 2022] DELETE /identity/v3/users/5286c3dbfb9d4aa29221094af2eca65c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:10.378297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:10.379872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:10.394450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3f862-651a-4c5a-8efa-a90e66e8c174 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:10.443920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:10.476065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:10.480671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:10.480671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:10.480671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] PATH_INFO: `/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:10.480671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eafd3cd20b044298416b2b0ac8fb0da)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:10.483079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:10.590697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eafd3cd20b044298416b2b0ac8fb0da', 'request_id': 'req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:10.657363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eafd3cd20b044298416b2b0ac8fb0da', 'request_id': 'req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:10.665569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eafd3cd20b044298416b2b0ac8fb0da', 'request_id': 'req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:10.682213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] The token cache is being invalidate because project 5eafd3cd20b044298416b2b0ac8fb0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:10.682569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:10.683684 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-9c4fc994-03c8-45cb-b7d1-3952e2073f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:10.688849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1196/2402] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:10 2022] DELETE /identity/v3/projects/5eafd3cd20b044298416b2b0ac8fb0da => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:10.690497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:10.690984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:10.691085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:10.714149 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:10.715368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:10.786539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1207/2403] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:10.788618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:10.788910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:10.789227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:10.815021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:10.816314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:10.826087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1197/2404] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:10.830279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:10.841714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed8d4-0400-406b-a6ed-d366107c3cbb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:10.846929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:10.847556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:10.847836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:10.848110 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:10.848817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:44:10.849157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:10.850752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:10.868544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1208/2405] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:44:10.872928 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:10.883309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5174f5-b63b-453a-98c1-df89a2d8ed40 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:10.890006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:10.890770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:10.891177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:10.891555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:10.892328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:10.894860 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:10.997469 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1198/2406] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:10 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:11.001720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.013296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bdf1d-ec94-4edc-a88e-402b8d125951 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.034071 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:11.074610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.075339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.075691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.076040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.076785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.078880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.214470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1209/2407] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:11 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:11.218488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.230048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acd012-4d6a-47ce-acb3-1dc9f01c686e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.234878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.235493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.236748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.236748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.236748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.238384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.252333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1199/2408] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:44:11.254416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.265385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6a64-7505-4c0c-a3df-f081b6d2f9fe admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.270297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.270917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.271197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.271470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.272179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.274083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590382-74b0-4f0e-a6df-7e568594519f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.285658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1210/2409] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:44:11.290605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.301039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadc28e-98a1-4fcd-913a-34497b9e4531 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.306191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.306886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.307168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.307447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] PATH_INFO: `/v3/projects/9f5c0310221048eea0ca999325cc5b5d/users/df5b822b61744f2287acace9cf9cb78d/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.321969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5c0310221048eea0ca999325cc5b5d, user_id=df5b822b61744f2287acace9cf9cb78d, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.324611 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.406420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1200/2410] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:11 2022] PUT /identity/v3/projects/9f5c0310221048eea0ca999325cc5b5d/users/df5b822b61744f2287acace9cf9cb78d/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:11.417591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.417881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.418161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.446968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] MFA Rules not processed for user `df5b822b61744f2287acace9cf9cb78d`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:11.448184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:11.510893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1211/2411] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:11.515232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.576318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb700382-b0cd-45b9-8345-0391f7f502df admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.580856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.581483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.581762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.582035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.582695 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.584599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.594725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:11.703996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1201/2412] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:11 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:11.708087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.719385 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2fc04-acf2-4282-aaed-93bb7dfbf854 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.775683 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.776301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.776582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.776856 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.777546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.779465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.850201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1212/2413] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:11 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:11.854283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.864645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3554-028b-4404-a175-9a0d7d7bea06 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.869496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.870102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.870388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.870665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.871367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.872961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.884158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1202/2414] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:11.888506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:11.898596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abad82-372f-441d-9008-259ed908c711 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:11.903902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:11.904559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:11.904839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:11.905153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] PATH_INFO: `/v3/projects/2b5129d591ba4063b90b59095f3901a9/users/cd1aba03338840e78218e67da0bc1679/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:11.919777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5129d591ba4063b90b59095f3901a9, user_id=cd1aba03338840e78218e67da0bc1679, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:11.922568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:11.925827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:12.039268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1213/2415] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:11 2022] PUT /identity/v3/projects/2b5129d591ba4063b90b59095f3901a9/users/cd1aba03338840e78218e67da0bc1679/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:12.043430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:12.053670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258cb1b-2029-46f8-9afe-f087786a56e1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:12.111002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:12.111639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:12.111921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:12.112190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:12.112877 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:12.114512 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9064b-1f38-40a2-84c6-b2b23b86b4fd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:12.124325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1203/2416] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:12.128594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:12.139812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dabd0e-fe8c-4ec4-a6a7-2a829a13eef0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:12.144386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:12.145070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:12.145349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:12.145620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] PATH_INFO: `/v3/projects/2b5129d591ba4063b90b59095f3901a9/users/cd1aba03338840e78218e67da0bc1679/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:12.147531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5129d591ba4063b90b59095f3901a9, user_id=cd1aba03338840e78218e67da0bc1679, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:12.151245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:12.164156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1214/2417] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:12 2022] PUT /identity/v3/projects/2b5129d591ba4063b90b59095f3901a9/users/cd1aba03338840e78218e67da0bc1679/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:12.233947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:12.234228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:12.234492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:12.262088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] MFA Rules not processed for user `cd1aba03338840e78218e67da0bc1679`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:12.263305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:12.331867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1204/2418] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:12.343205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:12.344850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:12.404289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a31be-7d50-4cec-b749-f82153e65eb4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:12.407948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:12.408648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:12.409125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:12.409368 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:12.418966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:12.420461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e132d2-33e7-4ee2-9241-fa4b17a6eae7 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:12.430571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1215/2419] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:13.410615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:13.472590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1b63f-b56c-4a01-a90d-5340944e3a5c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:13.477096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:13.478058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:13.478341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:13.479204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] PATH_INFO: `/v3/users/df5b822b61744f2287acace9cf9cb78d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:13.480369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5b822b61744f2287acace9cf9cb78d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:13.482307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:13.483825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:13.654723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5b822b61744f2287acace9cf9cb78d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:13.667586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5b822b61744f2287acace9cf9cb78d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:13.735953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5b822b61744f2287acace9cf9cb78d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:13.743640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5b822b61744f2287acace9cf9cb78d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:13.744274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5b822b61744f2287acace9cf9cb78d'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:13.750366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1205/2420] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:13 2022] DELETE /identity/v3/users/df5b822b61744f2287acace9cf9cb78d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:13.754895 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:13.755380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:13.755490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:13.775661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:13.776871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:13.834489 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1216/2421] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:13.844819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:13.846256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:13.903908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480fc2-3b19-4a61-b072-121d32aaf014 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:13.907687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:13.908304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:13.908588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:13.908867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:13.918637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:13.919996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:13.929694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1206/2422] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:14.298195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:14.299627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:14.314083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca743-1af9-4272-b268-e60e173a1743 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:14.318805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:14.319569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:14.319854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:14.320225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] PATH_INFO: `/v3/projects/9f5c0310221048eea0ca999325cc5b5d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:14.321442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f5c0310221048eea0ca999325cc5b5d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:14.323985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:14.443304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f5c0310221048eea0ca999325cc5b5d', 'request_id': 'req-cee0cc15-324e-45dc-9754-f7d5939879e1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.449629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f5c0310221048eea0ca999325cc5b5d', 'request_id': 'req-cee0cc15-324e-45dc-9754-f7d5939879e1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.504507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f5c0310221048eea0ca999325cc5b5d', 'request_id': 'req-cee0cc15-324e-45dc-9754-f7d5939879e1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.515367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] The token cache is being invalidate because project 9f5c0310221048eea0ca999325cc5b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:14.515669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.516268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.517928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1217/2423] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:14 2022] DELETE /identity/v3/projects/9f5c0310221048eea0ca999325cc5b5d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:14.521922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:14.531486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24dfb3-4dc2-44d9-a170-dc83f4691494 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:14.587990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:14.588630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:14.588914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:14.589223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] PATH_INFO: `/v3/users/cd1aba03338840e78218e67da0bc1679` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:14.590397 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1aba03338840e78218e67da0bc1679)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:14.592319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:14.615022 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:14.747187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1aba03338840e78218e67da0bc1679'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.759244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1aba03338840e78218e67da0bc1679'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.807898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1aba03338840e78218e67da0bc1679'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.815060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1aba03338840e78218e67da0bc1679'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.815685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0e1a14a0-de5e-4227-97e2-b5c9d1f91c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1aba03338840e78218e67da0bc1679'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:14.821675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1207/2424] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:14 2022] DELETE /identity/v3/users/cd1aba03338840e78218e67da0bc1679 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:44:15.226773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:15.228870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:15.245604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0cc15-324e-45dc-9754-f7d5939879e1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:15.250463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:15.311649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:15.312242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.312525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.312800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] PATH_INFO: `/v3/projects/2b5129d591ba4063b90b59095f3901a9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.314001 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5129d591ba4063b90b59095f3901a9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:15.315909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:15.431013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5129d591ba4063b90b59095f3901a9', 'request_id': 'req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:15.437283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5129d591ba4063b90b59095f3901a9', 'request_id': 'req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:15.513266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5129d591ba4063b90b59095f3901a9', 'request_id': 'req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:15.524094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] The token cache is being invalidate because project 2b5129d591ba4063b90b59095f3901a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:15.524378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:15.525649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-f589f6ef-d3b5-470d-a59b-7bf7b640bf0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:15.527186 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1218/2425] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:15 2022] DELETE /identity/v3/projects/2b5129d591ba4063b90b59095f3901a9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:15.532817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f3715-cf5f-4b00-806c-15385958807c None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.533364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f3715-cf5f-4b00-806c-15385958807c None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.533460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f3715-cf5f-4b00-806c-15385958807c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.553317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-651f3715-cf5f-4b00-806c-15385958807c None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:15.554509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-651f3715-cf5f-4b00-806c-15385958807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:15.620324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1208/2426] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:15.620607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.620878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.621185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.641834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:15.643019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:15.651048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1219/2427] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:15.655390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f3715-cf5f-4b00-806c-15385958807c None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:15.664978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f3715-cf5f-4b00-806c-15385958807c None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:15.670510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:15.671160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.671445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.671725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.672428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:44:15.672737 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:15.674741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:15.687051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1209/2428] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:44:15.691142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:15.700551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8a43d-28ba-4266-8cc2-59c0c0447ebe None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:15.705131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:15.705727 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.706004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.706275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.706921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:15.708801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:15.795503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1220/2429] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:15 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:15.804982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:15.816380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e9fcc-7511-4ff8-af8d-4b961ac70778 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:15.874230 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:15.874762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:15.875040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:15.875314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:15.875995 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:15.877954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:15.987946 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1210/2430] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:15 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:15.992160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.002744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cb52-5e9e-47b7-ad43-ddf347c49b06 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:16.007329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:16.007938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.008220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.008496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.009317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:16.010958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:16.021682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1221/2431] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:16.026427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.036508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f44ff-a0fe-4456-875e-cddbdebbe1c7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:16.042108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:16.042753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.043120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.044264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.044264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:16.045831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986481a-50db-420a-9fad-a286cd8727fd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:16.057546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1211/2432] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:16.062008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.071924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75ba64-d47a-4bd3-9d4d-79025789a069 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:16.076540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:16.077242 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.077562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.077804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] PATH_INFO: `/v3/projects/33e0bb52a5744807834308222ca38d82/users/7cc4aca0ee014780a7366aed7c204899/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.093332 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e0bb52a5744807834308222ca38d82, user_id=7cc4aca0ee014780a7366aed7c204899, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:16.096023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:16.193825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1222/2433] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:16 2022] PUT /identity/v3/projects/33e0bb52a5744807834308222ca38d82/users/7cc4aca0ee014780a7366aed7c204899/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:16.205321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.205613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.205913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.220225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:16.235570 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] MFA Rules not processed for user `7cc4aca0ee014780a7366aed7c204899`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:16.236817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:16.306557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1212/2434] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:16.317661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.319369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:16.360098 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:16.377981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc21fc-c795-4336-8ac6-1aa07f0c2743 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:16.381824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:16.382836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.383569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.383985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.393705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:16.394909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bcc05-3791-449a-b763-47c3324b09bb service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:16.405353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1223/2435] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:16.434452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.498762 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaad14-b246-46eb-97e5-c40c992a170b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:16.503566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:16.504203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.504652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.504807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] PATH_INFO: `/v3/users/7cc4aca0ee014780a7366aed7c204899` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.505947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc4aca0ee014780a7366aed7c204899)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:16.507855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:16.735238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc4aca0ee014780a7366aed7c204899'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:16.747778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc4aca0ee014780a7366aed7c204899'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:16.822882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc4aca0ee014780a7366aed7c204899'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:16.830365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc4aca0ee014780a7366aed7c204899'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:16.831004 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc4aca0ee014780a7366aed7c204899'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:16.837039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1213/2436] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:16 2022] DELETE /identity/v3/users/7cc4aca0ee014780a7366aed7c204899 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:16.841720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:16.842003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:16.842279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:16.860622 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:16.861873 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:16.926128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1224/2437] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:16.940823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:16.942297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:16.965122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:17.007822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc23d6d-2181-46d1-a633-35e8d452d166 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:17.011733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:17.012295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:17.012579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:17.012856 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:17.022190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:17.023844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cfeaf-985b-40ff-86cf-5430d0933a69 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:17.034040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1214/2438] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:17.638291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:17.639925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:17.654819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578723d-900f-4169-a48e-ff6dc7486d1a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:17.662331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:17.663562 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:17.664038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:17.664653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] PATH_INFO: `/v3/projects/33e0bb52a5744807834308222ca38d82` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:17.666067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e0bb52a5744807834308222ca38d82)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:17.671531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:17.776402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e0bb52a5744807834308222ca38d82', 'request_id': 'req-e8534fc7-f6ec-4a3e-8c31-5209a0975322'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:17.782702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e0bb52a5744807834308222ca38d82', 'request_id': 'req-e8534fc7-f6ec-4a3e-8c31-5209a0975322'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:17.856415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e0bb52a5744807834308222ca38d82', 'request_id': 'req-e8534fc7-f6ec-4a3e-8c31-5209a0975322'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:17.869643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] The token cache is being invalidate because project 33e0bb52a5744807834308222ca38d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:17.869958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:17.870583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e8534fc7-f6ec-4a3e-8c31-5209a0975322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:17.872106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1225/2439] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:17 2022] DELETE /identity/v3/projects/33e0bb52a5744807834308222ca38d82 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:44:17.877031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:17.877506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:17.877600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:17.898525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:17.899643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:17.960506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1215/2440] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:17.969198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:17.969505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:17.969767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:17.993178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:17.995005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:18.002951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1226/2441] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:18.023427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.033548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b00965-3d9f-4149-b2eb-3cdce2632131 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.038228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.038839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.039123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.039401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.040096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:44:18.040400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.042642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.054534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1216/2442] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:44:18.059804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.069530 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9fd57-12b1-4d95-bfe5-d928c57e75b1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.074512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.075118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.075394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.075665 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.076333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.078330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.170507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1227/2443] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:44:18 2022] POST /identity/v3/projects => generated 359 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:44:18.175087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.185482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7a49-9aa4-4d67-97d0-ab5a2b9cf3ac admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.241978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.242599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.242881 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.243158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.243976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.245940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.333349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1217/2444] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:44:18 2022] POST /identity/v3/users => generated 280 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:44:18.338046 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.348483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162e8b9-db83-493b-9d3e-11b8d33aad5c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.353704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.354480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.354910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.355174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.355824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.357457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.368053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1228/2445] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:18.372361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.382768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bda1-b5d6-4150-8390-cb446ca5eba7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.388001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.388642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.389048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.389442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.390168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.391801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039ef5f-eb62-45a2-8d40-4fb6ab5c2b15 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.403410 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1218/2446] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:44:18.407818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.418785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d1a-ad22-4921-a275-6d9d0d6a1560 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.423634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.429175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.429175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.429175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] PATH_INFO: `/v3/projects/7c2ba3f982ee447c85d264884ce5607a/users/75fe7b1ae11047bf97542db9da5e8f2f/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.439784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2ba3f982ee447c85d264884ce5607a, user_id=75fe7b1ae11047bf97542db9da5e8f2f, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.442463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.505047 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:18.511281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1229/2447] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:44:18 2022] PUT /identity/v3/projects/7c2ba3f982ee447c85d264884ce5607a/users/75fe7b1ae11047bf97542db9da5e8f2f/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:18.521665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.521978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.522256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.551763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] MFA Rules not processed for user `75fe7b1ae11047bf97542db9da5e8f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:18.552948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:18.564406 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:18.617102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1219/2448] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:18.652336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.654083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:18.711616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9a34-ddbf-4c1f-aa57-3afe8a48fa34 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.716278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.716976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.717349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.717688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.726359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.729091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb17f73-e75c-4086-8788-7188ad6efc20 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.742798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1230/2449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:18.760817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:18.827729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676b32-9bd2-4b76-9eb1-445fa69bcb43 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:18.832618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:18.833283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:18.833571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:18.833852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] PATH_INFO: `/v3/users/75fe7b1ae11047bf97542db9da5e8f2f` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:18.835039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fe7b1ae11047bf97542db9da5e8f2f)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:18.837016 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:18.995744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fe7b1ae11047bf97542db9da5e8f2f'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.008659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fe7b1ae11047bf97542db9da5e8f2f'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.067592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fe7b1ae11047bf97542db9da5e8f2f'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.075213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fe7b1ae11047bf97542db9da5e8f2f'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.078383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fe7b1ae11047bf97542db9da5e8f2f'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.082367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1220/2450] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:44:18 2022] DELETE /identity/v3/users/75fe7b1ae11047bf97542db9da5e8f2f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:19.087108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:19.087963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:19.088239 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:19.108381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:19.109627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:19.175324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1231/2451] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:19.185164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:19.186651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:19.221879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:19.244573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c318a5-6ffa-49a6-994e-c44827f003cc admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:19.248300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:19.248853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:19.249169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:19.249438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:19.258710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:19.261109 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:19.270799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1221/2452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:19.785926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:19.785926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:19.794373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d001022-f1a6-433e-880f-860ea5943753 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:19.799992 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:19.800857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:19.801182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:19.801459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] PATH_INFO: `/v3/projects/7c2ba3f982ee447c85d264884ce5607a` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:19.802751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2ba3f982ee447c85d264884ce5607a)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:19.805449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:19.948539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2ba3f982ee447c85d264884ce5607a', 'request_id': 'req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:19.955035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2ba3f982ee447c85d264884ce5607a', 'request_id': 'req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:20.016420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2ba3f982ee447c85d264884ce5607a', 'request_id': 'req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:20.027409 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] The token cache is being invalidate because project 7c2ba3f982ee447c85d264884ce5607a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:44:20.027640 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:20.028268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ec4417a9-5ae4-4772-9f3f-aa63ce1688ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:44:20.029799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1232/2453] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:44:19 2022] DELETE /identity/v3/projects/7c2ba3f982ee447c85d264884ce5607a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:44:29.833378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=11 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:30.024397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:35.517438 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:44:35.518879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:35.597915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6a532-18dc-47b0-aeb5-550ce02d32a5 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:44:35.670029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:44:35.670029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:35.670029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:35.670029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:35.670558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:35.679455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:44:35.681050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442759a2-b378-4300-bc84-5350701b1db0 service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:44:35.691650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1222/2454] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:44:40.417137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:44:40.417657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:44:40.417763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:44:40.438181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] MFA Rules not processed for user `30432a068ad8486db8af635e66776f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:44:40.439380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:44:40.493175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1233/2455] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:44:45.687332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=58 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:44:50.486341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=25 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:01.083490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9d03c-ba88-4b63-9060-9bf705b600ea None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:01.089244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9d03c-ba88-4b63-9060-9bf705b600ea None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:01.089244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9d03c-ba88-4b63-9060-9bf705b600ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:01.104564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9ed9d03c-ba88-4b63-9060-9bf705b600ea None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:45:01.105784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ed9d03c-ba88-4b63-9060-9bf705b600ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:01.162258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1223/2456] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:45:01.296569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:01.298266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:01.368293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa2b2-5aa4-44d2-801e-b18306283e12 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:01.373518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:01.374131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:01.374408 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:01.374682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:01.384258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:01.385880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:01.394577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1234/2457] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:45:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:45:11.153705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=25 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:11.391831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=32 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:21.225498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:21.225961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:21.226073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:21.245036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:45:21.246203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:21.253924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1224/2458] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:45:21.266322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:21.276691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a1a7-17cc-41aa-b228-a7639d43b696 service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:21.282301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:21.282906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:21.283184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:21.283459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:21.291391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:21.292959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:21.301989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1235/2459] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:45:31.239734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:31.298573 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:33.077876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:33.079267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:33.134709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a538d7-022d-4d36-be05-44e9190bbcd7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:33.138426 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:33.139038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:33.139343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:33.139613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:33.141367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:33.224942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:33.226413 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:33.246989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1225/2460] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:45:34.434620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:34.436038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:34.448583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fd0ca-825d-478d-a271-61fabd4e9ec3 service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:34.452139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:34.452740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:34.453057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:34.453337 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:34.455018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:34.468380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:34.469823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e0b73-d39f-4089-ae8b-17979053f0e0 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:34.481308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1236/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:45:42.668939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:42.670578 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:42.685385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e878fa1-47aa-46d9-95f0-ba073f1c2031 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:42.690921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:42.691675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:42.691962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:42.692244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:42.694034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:42.776805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:42.778422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:42.782882 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:42.797649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1226/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:45:43.143817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:43.144896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:43.144896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:43.165312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:45:43.166511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:45:43.173942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1237/2463] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:45:43.186657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:45:43.198285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88caa7-76bd-4098-9065-6c903d07739a service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:45:43.203488 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:45:43.204096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:45:43.204399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:45:43.204693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:45:43.214947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:45:43.216511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d06a0-9ffd-451e-9a8c-c41c0896a999 service swift] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:45:43.224797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1227/2464] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:45:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 14:45:53.159348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=9 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:45:53.221272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=10 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:13.702500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:13.703909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:13.759938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c98409-a8de-4489-95de-9a4de695ff11 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:13.763525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:13.764134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:13.764414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:13.764688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:13.766453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:13.828201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:13.830018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:13.839945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1238/2465] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:23.835198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=48 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:25.328583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:25.329043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:25.329182 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:25.349203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] MFA Rules not processed for user `f9360a76354e4ae99668fe8debe4e7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:25.350402 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:25.357323 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1228/2466] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:26.974250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:26.984394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459efb-336d-4c50-a7b6-89d18a83f9b9 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:26.988292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:26.988976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:26.989388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:26.989725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:26.991565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:27.058207 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:27.059786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:27.069827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1239/2467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:30.197345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:30.207416 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83140f70-d4cb-4657-9d6b-aef9b3777e6f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:30.211082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:30.211696 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:30.211977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:30.212255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:30.214164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:30.224569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:30.226045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:30.234690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1229/2468] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:30.967738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:30.969936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:30.983539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59b820-9d75-4fc7-85d7-00d92d95f948 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:30.988431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:30.989128 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:30.989413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:30.989690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] PATH_INFO: `/v3/users/f3a38dc21bc34164aea48065da64823f` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:31.006205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a38dc21bc34164aea48065da64823f)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:31.008179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:31.236118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a38dc21bc34164aea48065da64823f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.248982 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a38dc21bc34164aea48065da64823f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.257339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a38dc21bc34164aea48065da64823f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.258054 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a38dc21bc34164aea48065da64823f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.336243 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-31c2dd52-74de-43ae-ae2d-a14170d2324f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a38dc21bc34164aea48065da64823f'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.343502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1240/2469] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:46:30 2022] DELETE /identity/v3/users/f3a38dc21bc34164aea48065da64823f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:31.594968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:31.608623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935e7b8-27b3-424f-885c-69eed403138e service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:31.669524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:31.670303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:31.670586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:31.670855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] PATH_INFO: `/v3/projects/44c869dc06224bf49f25fcf7aeb0e5a7` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:31.682266 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c869dc06224bf49f25fcf7aeb0e5a7)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:31.684236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:31.802141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c869dc06224bf49f25fcf7aeb0e5a7', 'request_id': 'req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.872062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c869dc06224bf49f25fcf7aeb0e5a7', 'request_id': 'req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.883993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c869dc06224bf49f25fcf7aeb0e5a7', 'request_id': 'req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.895648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] The token cache is being invalidate because project 44c869dc06224bf49f25fcf7aeb0e5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:46:31.896228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.897102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5a73cd40-7b50-40ce-a29e-e1f0da7c2b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:31.899070 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1230/2470] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:46:31 2022] DELETE /identity/v3/projects/44c869dc06224bf49f25fcf7aeb0e5a7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:31.903903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:31.904187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:31.904460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:31.938192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:31.939396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:31.988955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:31.999730 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1241/2471] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:32.007528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.007982 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.008270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.028476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:32.029874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:32.038731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1231/2472] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:32.042913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.053857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f5f4-29aa-42ba-b9d0-8330398f1408 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.058467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.059069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.059343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.059616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.060304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:46:32.060606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.062215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.073834 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1242/2473] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:46:32.078038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.088379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929ca11-dc45-4e07-b67f-73ca29ce77f0 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.093237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.093845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.094128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.094404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.095112 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.097618 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.250715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1232/2474] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:46:32 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:46:32.255036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.265130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073595cd-9296-474e-ad88-9e784812b68c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.324909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.325583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.325864 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.326138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.326803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.328697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.582084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1243/2475] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:46:32 2022] POST /identity/v3/users => generated 292 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:46:32.586302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.596453 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbd236-a2a6-44f4-b3e6-d55ffa5065fd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.601665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.602265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.602569 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.602844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.603589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.605221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.615661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1233/2476] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:32.619627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.629811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04be0a3-beec-4943-8deb-b20ae04fd83f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.634589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.635222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.635501 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.635774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.636496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.638181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4cf1a-cc7e-4a34-af09-c79c42cf8042 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.649201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1244/2477] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:32.653144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.663600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d560645-68b6-49b0-94c2-fc8f36bbc98d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.668127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.668773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.669089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.669365 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] PATH_INFO: `/v3/projects/97a611dec40d40bdbc0c21e2223793e5/users/7687d5ac872449efb74266fe41dfb9eb/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.681268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:32.685274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a611dec40d40bdbc0c21e2223793e5, user_id=7687d5ac872449efb74266fe41dfb9eb, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.688010 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.776840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1234/2478] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:46:32 2022] PUT /identity/v3/projects/97a611dec40d40bdbc0c21e2223793e5/users/7687d5ac872449efb74266fe41dfb9eb/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:32.789445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.789729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.790004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.821769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] MFA Rules not processed for user `7687d5ac872449efb74266fe41dfb9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:32.821769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:32.883962 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1245/2479] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:32.895229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:32.896906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:32.974506 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844289c-a92b-4a08-916c-1dba112656e6 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:32.980061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:32.980674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:32.980955 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:32.981256 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:32.989200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:32.990819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a63c01-c272-400c-a883-dbc6562254d3 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:32.999482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1235/2480] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:46:42.881114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:42.995458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:43.287415 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:43.351717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c32fe-af41-4d66-9043-22082c44d447 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:43.356617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:43.357300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:43.357582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:43.357862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] PATH_INFO: `/v3/users/7687d5ac872449efb74266fe41dfb9eb` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:43.359059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7687d5ac872449efb74266fe41dfb9eb)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:43.361034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:43.524819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7687d5ac872449efb74266fe41dfb9eb'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:43.537822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7687d5ac872449efb74266fe41dfb9eb'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:43.544635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7687d5ac872449efb74266fe41dfb9eb'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:43.545648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7687d5ac872449efb74266fe41dfb9eb'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:43.616118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7687d5ac872449efb74266fe41dfb9eb'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:43.622975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1246/2481] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:46:43 2022] DELETE /identity/v3/users/7687d5ac872449efb74266fe41dfb9eb => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:43.627644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:43.627933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:43.628212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:43.648093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:43.652144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:43.709815 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1236/2482] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:43.719679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:43.721141 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:43.780366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622188c9-4f19-49f7-b66d-93f0a45ac76b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:43.783953 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:43.786516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:43.786516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:43.786516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:43.795674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:43.797569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a31b95-01a5-4a6b-9258-d0a48b8117e2 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:43.808563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1247/2483] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:44.229139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.229821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:44.244879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3c363-4b1f-4184-948c-dcd8cf2e96d3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.250854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.251495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.251824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.252025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] PATH_INFO: `/v3/projects/97a611dec40d40bdbc0c21e2223793e5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.253313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a611dec40d40bdbc0c21e2223793e5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.255630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.376287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a611dec40d40bdbc0c21e2223793e5', 'request_id': 'req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:44.449830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a611dec40d40bdbc0c21e2223793e5', 'request_id': 'req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:44.456139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a611dec40d40bdbc0c21e2223793e5', 'request_id': 'req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:44.467232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] The token cache is being invalidate because project 97a611dec40d40bdbc0c21e2223793e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:46:44.467517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:44.468140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f0a9d6ac-e9e7-40b6-8dd0-530831434b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:44.469784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1237/2484] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:46:44 2022] DELETE /identity/v3/projects/97a611dec40d40bdbc0c21e2223793e5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:44.474909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.475380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.475486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.495810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:44.497007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:44.513971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:44.556636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1248/2485] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:46:44.564404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.564710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.564974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.586098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:44.587305 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:44.595215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1238/2486] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:44.599528 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.608973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97d9b-6967-4fb2-a183-d7b042f966dc None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.613411 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.614329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.614815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.615246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.616296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:46:44.616625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.618304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.630666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1249/2487] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:46:44.634813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.644043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a286dad-06b2-4a9c-8d4a-36a146353a7b None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.649604 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.650213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.650492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.650767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.651435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.653359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.728560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1239/2488] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:46:44 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:46:44.732722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.742986 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b3bb-00e0-476f-8119-61a3af2edb7f admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.800359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.800970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.801806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.802078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.802762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.804662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.871213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1250/2489] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:46:44 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:46:44.875322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.887278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61451833-c691-45a0-a788-7dc386e0b772 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.891819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.892480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.892781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.893060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.893760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.895565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.906041 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1240/2490] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:44.910145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.920788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbf16a-5bb2-4aff-8b60-b905d994df6e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.926617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.927475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.927754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.928025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.928720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.930395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade26dad-6678-4a47-9bc8-64c105fbf0cd admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:44.941628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1251/2491] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:44.947086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:44.962934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112cb65-3617-4ab0-9fdf-9299b94daaab admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:44.967657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:44.968345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:44.968616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:44.968895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] PATH_INFO: `/v3/projects/aa231925ae71487099fc319505f1bf1d/users/decf3115a6a34608a55a000ec40c7add/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:44.984237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa231925ae71487099fc319505f1bf1d, user_id=decf3115a6a34608a55a000ec40c7add, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:44.986914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:45.061642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1241/2492] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:46:44 2022] PUT /identity/v3/projects/aa231925ae71487099fc319505f1bf1d/users/decf3115a6a34608a55a000ec40c7add/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:46:45.074383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:45.074670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:45.074949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:45.102524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] MFA Rules not processed for user `decf3115a6a34608a55a000ec40c7add`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:45.103728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:45.168649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1252/2493] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:45.180690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:45.182447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:45.204181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:45.240102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6d1fc-5f3c-4a75-a2a3-0ec25b7c2bba admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:45.243859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:45.244479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:45.244761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:45.245174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:45.253661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:45.255378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d2a8d-552d-4572-abe8-530d31bd35d8 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:45.264921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1242/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:45.284272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:45.305605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:45.350999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9037c6-b2a6-468e-b416-ecc7de227d0d None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:45.355845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:45.356522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:45.356867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:45.357180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] PATH_INFO: `/v3/users/decf3115a6a34608a55a000ec40c7add` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:45.358380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=decf3115a6a34608a55a000ec40c7add)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:45.360420 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:45.595453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'decf3115a6a34608a55a000ec40c7add'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:45.607592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'decf3115a6a34608a55a000ec40c7add'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:45.614965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'decf3115a6a34608a55a000ec40c7add'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:45.615796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'decf3115a6a34608a55a000ec40c7add'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:45.718513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'decf3115a6a34608a55a000ec40c7add'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:45.724753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1253/2495] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:46:45 2022] DELETE /identity/v3/users/decf3115a6a34608a55a000ec40c7add => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:45.729328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:45.729621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:45.729899 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:45.748934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:45.750166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:45.808729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1243/2496] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:45.819348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:45.820732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:45.876618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232df8-881d-4ec6-8e9f-388984cfb7ca admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:45.880789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:45.881431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:45.881717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:45.881995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:45.891018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:45.892455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb133bc4-c3af-416d-bbe4-5bed3765a37c service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:45.901883 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1254/2497] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:46.377462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:46.378898 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:46.392373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b1550-b58e-40a7-9a2b-fc8c21c2295f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:46.409064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:46.410327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.410619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.410897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] PATH_INFO: `/v3/projects/aa231925ae71487099fc319505f1bf1d` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.414930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa231925ae71487099fc319505f1bf1d)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:46.416864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:46.525883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa231925ae71487099fc319505f1bf1d', 'request_id': 'req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:46.587485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa231925ae71487099fc319505f1bf1d', 'request_id': 'req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:46.594492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa231925ae71487099fc319505f1bf1d', 'request_id': 'req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:46.605914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] The token cache is being invalidate because project aa231925ae71487099fc319505f1bf1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:46:46.606206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:46.606865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1c57833f-e83a-4a0b-95ab-620e6ee9eaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:46:46.608423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1244/2498] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:46:46 2022] DELETE /identity/v3/projects/aa231925ae71487099fc319505f1bf1d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:46.613390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.613896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.614005 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.636017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:46.637439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:46.672970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:46.701950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1255/2499] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:46:46.709857 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.710152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.710435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.731193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:46.735091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:46.744424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1245/2500] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:46:46.748300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:46.759299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af06e6-0cc4-46a1-bbe1-abef0e7e77e6 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:46.764066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:46.764682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.764958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.769324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.769324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:46:46.769324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:46.769324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:46.780358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1256/2501] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:46:46.785008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:46.794747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2874a-d5dc-4dc1-bcba-968f910e898f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:46.801020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:46.801778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.802840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.803424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.803866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:46.805963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:46.888031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1246/2502] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:46:46 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:46:46.889582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:46.900063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b946-45c9-4a52-859e-9f6dd8d8071d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:46.960720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:46.961376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:46.961661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:46.961937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:46.962594 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:46.964921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:47.106620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1257/2503] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:46:46 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:46:47.107968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:47.119559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2605ba-cbfe-4bcd-adeb-e99e0b5cc522 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:47.124215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:47.124846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:47.125169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:47.125452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:47.126162 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:47.127791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:47.139528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1247/2504] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:47.143744 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:47.154679 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35208ae6-59f9-4746-b6f6-a776cb237eb0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:47.160040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:47.160849 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:47.161161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:47.161454 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:47.162827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:47.164463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe3c9e-dd2c-4a96-a1dd-52a69a909f50 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:47.178376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1258/2505] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:46:47.182823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:47.193456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffaa01-1f82-45ce-bd26-c492c9e83172 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:47.198646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:47.199329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:47.199613 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:47.199889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] PATH_INFO: `/v3/projects/ce222e775c7c4247a7a7d370deb5b2d0/users/2d7fa6c685204593ac573e23170b5a75/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:47.202499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:47.216559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce222e775c7c4247a7a7d370deb5b2d0, user_id=2d7fa6c685204593ac573e23170b5a75, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:47.219280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:47.313050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1248/2506] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:46:47 2022] PUT /identity/v3/projects/ce222e775c7c4247a7a7d370deb5b2d0/users/2d7fa6c685204593ac573e23170b5a75/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:46:47.315124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:47.315564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:47.316018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:47.347351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] MFA Rules not processed for user `2d7fa6c685204593ac573e23170b5a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:46:47.348556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:47.431516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1259/2507] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:46:47.444942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:47.449100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:47.512321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ece1e-5c25-43a5-8b22-ef68311d6493 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:47.517233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:47.517233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:47.517439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:47.517664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:47.532083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:47.533777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:47.546101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1249/2508] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:46:57.427450 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:57.541160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:46:58.711140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:46:58.718607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:58.769264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379b3b5-793f-492e-ab2f-019a6b219ea1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:46:58.772985 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:46:58.773624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:46:58.773904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:46:58.774181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:46:58.775913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:46:58.838900 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:46:58.840319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:46:58.849948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1260/2509] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:03.950837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:03.963149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15872544-ec69-4f39-b626-ed260eec7d47 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:03.966911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:03.967529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:03.967895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:03.968103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:03.969940 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:04.037055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:04.038685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:04.048809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1250/2510] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:07.705956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:07.707506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:07.721906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5be2ad-469d-4f31-a40a-02f80b5908b7 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:07.727271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:07.727914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:07.728193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:07.728466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] PATH_INFO: `/v3/users/30432a068ad8486db8af635e66776f90` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:07.742482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] RBAC: Authorizing `identity:delete_user(user_id=30432a068ad8486db8af635e66776f90)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:07.744417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:07.943660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30432a068ad8486db8af635e66776f90'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:07.955797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30432a068ad8486db8af635e66776f90'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:07.963367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30432a068ad8486db8af635e66776f90'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:07.964102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30432a068ad8486db8af635e66776f90'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.047314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-17f3514d-cbba-4e6c-b1fb-88e6bd350add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30432a068ad8486db8af635e66776f90'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.053629 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1261/2511] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:07 2022] DELETE /identity/v3/users/30432a068ad8486db8af635e66776f90 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:08.336079 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:08.342385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631958ff-cc8d-4a95-ac8e-87634c9a84d2 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:08.401715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:08.402362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.402645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.402921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] PATH_INFO: `/v3/projects/1776579056dd420d81368ec51e2ddb95` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.414489 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1776579056dd420d81368ec51e2ddb95)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:08.416449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:08.515905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1776579056dd420d81368ec51e2ddb95', 'request_id': 'req-7c186935-c802-411f-bbf4-b22f3f70a315'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.618428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1776579056dd420d81368ec51e2ddb95', 'request_id': 'req-7c186935-c802-411f-bbf4-b22f3f70a315'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.625387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1776579056dd420d81368ec51e2ddb95', 'request_id': 'req-7c186935-c802-411f-bbf4-b22f3f70a315'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.636417 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] The token cache is being invalidate because project 1776579056dd420d81368ec51e2ddb95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:08.636703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.637354 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c186935-c802-411f-bbf4-b22f3f70a315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:08.638904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1251/2512] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:08 2022] DELETE /identity/v3/projects/1776579056dd420d81368ec51e2ddb95 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:08.644104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.644394 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.644710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.667364 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:08.668603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:08.674061 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:08.727183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1262/2513] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:08.734710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.742583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.742583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.754745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:08.755941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:08.764383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1252/2514] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:08.768496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:08.778674 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977946b-281b-44e3-8026-a16ea3f12c31 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:08.783521 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:08.785124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.785124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.785124 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.785544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:08.785847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:08.787507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:08.799929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1263/2515] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:47:08.804967 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:08.814372 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62c2a-8073-42c0-b6f1-c045540147cb None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:08.819251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:08.820002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.820409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.820685 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.821415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:08.823345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:08.829020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:08.908428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1253/2516] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:08 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:08.912500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:08.923754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064197c3-a31a-45b6-9da8-398a6ce0f347 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:08.981183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:08.981796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:08.982078 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:08.982355 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:08.983013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:08.984925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:09.070911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1264/2517] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:08 2022] POST /identity/v3/users => generated 282 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:09.074934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:09.085324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fa037-ab18-4772-af41-44edab04f5fb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:09.090448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:09.091077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:09.091437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:09.091660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:09.092363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:09.094009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:09.105049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1254/2518] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:09.109157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:09.120100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa0275-eb77-4487-890a-27bd863e561e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:09.130439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:09.130439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:09.130439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:09.130439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:09.130835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:09.130835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527fb82-af67-414a-856e-541c89939dd7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:09.139016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1265/2519] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:09.143139 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:09.153740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645ef0e-144e-4348-b332-4c22b184c715 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:09.161368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:09.161368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:09.161368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:09.161368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] PATH_INFO: `/v3/projects/12036465613a459e81ba8d609b2b9d69/users/264dd6fac5af47ceb04afa4e4f6ad191/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:09.175750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12036465613a459e81ba8d609b2b9d69, user_id=264dd6fac5af47ceb04afa4e4f6ad191, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:09.179126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:09.284347 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1255/2520] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:09 2022] PUT /identity/v3/projects/12036465613a459e81ba8d609b2b9d69/users/264dd6fac5af47ceb04afa4e4f6ad191/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:09.296749 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:09.297067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:09.297346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:09.332135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] MFA Rules not processed for user `264dd6fac5af47ceb04afa4e4f6ad191`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:09.333373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:09.398292 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1266/2521] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:09.409496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:09.411193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:09.468303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d032e9-cdb5-4310-8bd1-b7271ccd064f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:09.471952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:09.472582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:09.472864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:09.473178 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:09.481781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:09.483387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:09.492755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1256/2522] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:09 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:17.854070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:17.855503 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:17.865403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:17.943117 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d2c56-f7c4-49fa-9006-7ad1da31510f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:17.951059 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:17.956286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:17.956286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:17.956286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:17.956286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:17.969092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:17.970745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa0b61-9a0a-406c-a666-3d2f1f8066f3 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:17.986607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1267/2523] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:19.488008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:21.131166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:21.132804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:21.199723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa8abb-fce8-49e8-9ea3-172248b0b958 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:21.204407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:21.205077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:21.205361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:21.205639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] PATH_INFO: `/v3/users/2d7fa6c685204593ac573e23170b5a75` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:21.206807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7fa6c685204593ac573e23170b5a75)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:21.208830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:21.374886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7fa6c685204593ac573e23170b5a75'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:21.386680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7fa6c685204593ac573e23170b5a75'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:21.458407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7fa6c685204593ac573e23170b5a75'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:21.466275 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7fa6c685204593ac573e23170b5a75'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:21.466921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7fa6c685204593ac573e23170b5a75'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:21.472914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1257/2524] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:21 2022] DELETE /identity/v3/users/2d7fa6c685204593ac573e23170b5a75 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:21.478260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:21.478745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:21.478846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:21.497406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:21.498597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:21.557290 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1268/2525] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:21.566912 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:21.568293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:21.624763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbbd8f-990a-46e9-a3b2-167371c3b122 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:21.628398 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:21.629011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:21.629313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:21.629584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:21.639336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:21.640767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc95de4-ca1a-4432-a28f-8b753208cdd2 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:21.651084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1258/2526] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:22.089475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.091011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:22.104624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecc4d7-5afb-49bf-b649-1692cc9390eb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.116306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.117154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.117474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.117707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] PATH_INFO: `/v3/projects/ce222e775c7c4247a7a7d370deb5b2d0` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.119050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce222e775c7c4247a7a7d370deb5b2d0)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.121142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.236700 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce222e775c7c4247a7a7d370deb5b2d0', 'request_id': 'req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:22.244574 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce222e775c7c4247a7a7d370deb5b2d0', 'request_id': 'req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:22.321949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce222e775c7c4247a7a7d370deb5b2d0', 'request_id': 'req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:22.334823 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] The token cache is being invalidate because project ce222e775c7c4247a7a7d370deb5b2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:22.335374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:22.337881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ba5d8dd2-908f-4ed2-9646-7f757d9b3d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:22.338070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1269/2527] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:22 2022] DELETE /identity/v3/projects/ce222e775c7c4247a7a7d370deb5b2d0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:22.343629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.344106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.344213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.367988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:22.369231 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:22.387970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:22.435344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1259/2528] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:22.443102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.443391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.444338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.464867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:22.466091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:22.473973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1270/2529] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:22.479532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.489446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3ced9-5daa-45a8-a872-5eb7f0be718e None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.494457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.495222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.495511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.495789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.496690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:22.496805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.498452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.511863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1260/2530] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:47:22.516129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.526399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaa3bf-331d-47bc-b45f-01c1873e58bf None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.531132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.531791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.532053 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.532327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.533041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.534967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.540618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:22.628260 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1271/2531] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:22.633382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.643394 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053d0-b14a-4142-a6c0-fa4884fbe401 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.707050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.707763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.708050 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.708329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.709051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.711006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.796526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1261/2532] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:22 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:22.802232 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.813361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643ea1-3e83-44e8-bb94-b665e5477197 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.818181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.819062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.819318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.819615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.820341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.821964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.832678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1272/2533] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:22.837462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.847825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1b35e-751c-4bf3-9517-a888e84bf14d admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.853310 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.853924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.854207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.854483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.855183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.856776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b60679-8e73-4e7b-8da9-65d703ac0214 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.867557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1262/2534] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:22.871634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:22.881798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072ae6-047d-4658-9378-be7de294c45a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:22.886495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:22.887153 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.887439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.887720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] PATH_INFO: `/v3/projects/fb55231013b4448ca78b749a09c5c3ff/users/f3424f871a224e8d92733a9556ce8284/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:22.903156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb55231013b4448ca78b749a09c5c3ff, user_id=f3424f871a224e8d92733a9556ce8284, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:22.906471 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:22.987134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1273/2535] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:22 2022] PUT /identity/v3/projects/fb55231013b4448ca78b749a09c5c3ff/users/f3424f871a224e8d92733a9556ce8284/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:22.997588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:22.997880 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:22.998161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:23.039001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] MFA Rules not processed for user `f3424f871a224e8d92733a9556ce8284`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:23.040191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:23.109370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:23.109949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1263/2536] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:23.111319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:23.122513 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:23.124037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:23.144317 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:23.174847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4953fc5-5ead-4182-8f27-3b3a985ff68d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:23.178772 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:23.179273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:23.179554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:23.179828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:23.181615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:23.186427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81e562-5194-4004-80b6-461105d65d52 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:23.190129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:23.190687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:23.190968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:23.195509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:23.200746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:23.202290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:23.211716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1264/2537] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:47:23.249894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:23.251305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628ef9-726f-4d96-9db5-e184b68d2c46 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:23.261984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1274/2538] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:47:26.650461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:26.652118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:26.666199 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fae6f-60ae-4713-be22-19ee6d3439d0 service glance] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:26.676932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:26.676932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:26.676932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:26.676932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] PATH_INFO: `/v3/users/264dd6fac5af47ceb04afa4e4f6ad191` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:26.677355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264dd6fac5af47ceb04afa4e4f6ad191)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:26.680085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:26.866263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264dd6fac5af47ceb04afa4e4f6ad191'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:26.878921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264dd6fac5af47ceb04afa4e4f6ad191'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:26.940553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264dd6fac5af47ceb04afa4e4f6ad191'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:26.948663 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264dd6fac5af47ceb04afa4e4f6ad191'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:26.950450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264dd6fac5af47ceb04afa4e4f6ad191'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:26.955653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1265/2539] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:26 2022] DELETE /identity/v3/users/264dd6fac5af47ceb04afa4e4f6ad191 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:26.961552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:26.962205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:26.962205 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:26.982467 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:26.983712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.027924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:27.029888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.062244 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1275/2540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:27.072115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:27.073529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.078836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:27.103399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37980-a62e-4fd1-8b67-82e57651e8c4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:27.109209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:27.109348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.109573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.109849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] PATH_INFO: `/v3/users/de769d721b8b49c6a5d7d391412da122` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.111030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de769d721b8b49c6a5d7d391412da122)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:27.112954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:27.136459 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774f4ce-0b14-4699-8c72-39a479367ecc None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:27.140202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:27.140819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.141135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.141418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.151075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:27.152578 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:27.168442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1276/2541] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:47:27.300707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:27.310652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de769d721b8b49c6a5d7d391412da122'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.324153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de769d721b8b49c6a5d7d391412da122'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.529750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de769d721b8b49c6a5d7d391412da122'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.537295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de769d721b8b49c6a5d7d391412da122'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.537968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de769d721b8b49c6a5d7d391412da122'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.543820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1266/2542] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:27 2022] DELETE /identity/v3/users/de769d721b8b49c6a5d7d391412da122 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:27.576396 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:27.577906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.591383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ca403-0968-4e2a-83de-0962976e3342 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:27.657187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:27.657958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.658391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.658651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] PATH_INFO: `/v3/projects/12036465613a459e81ba8d609b2b9d69` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.659912 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12036465613a459e81ba8d609b2b9d69)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:27.661870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:27.810089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12036465613a459e81ba8d609b2b9d69', 'request_id': 'req-c892326c-4421-4ef6-b4d6-157e44b25d48'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.816139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12036465613a459e81ba8d609b2b9d69', 'request_id': 'req-c892326c-4421-4ef6-b4d6-157e44b25d48'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.827297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:27.828808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.845161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1412cb-47ea-4a1d-9940-688dbe4f33e7 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:27.887317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12036465613a459e81ba8d609b2b9d69', 'request_id': 'req-c892326c-4421-4ef6-b4d6-157e44b25d48'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.900041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] The token cache is being invalidate because project 12036465613a459e81ba8d609b2b9d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:27.900407 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.901073 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-c892326c-4421-4ef6-b4d6-157e44b25d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:27.902645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1277/2543] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:27 2022] DELETE /identity/v3/projects/12036465613a459e81ba8d609b2b9d69 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:27.906005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:27.906005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.906257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.906427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] PATH_INFO: `/v3/projects/fdbc49e630fc461a858eb04c2dd52a82` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.907350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea25d-17b7-4f9e-9e87-11d4938f487b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.907615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbc49e630fc461a858eb04c2dd52a82)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:27.907718 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea25d-17b7-4f9e-9e87-11d4938f487b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.907891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea25d-17b7-4f9e-9e87-11d4938f487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.909600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:27.934842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-587ea25d-17b7-4f9e-9e87-11d4938f487b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:27.936092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-587ea25d-17b7-4f9e-9e87-11d4938f487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.944188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1278/2544] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:27.955193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:27.955193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:27.955193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:27.971995 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:27.973263 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:27.981913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1279/2545] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:27.986041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:27.995719 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7540a-1512-4d59-b5c4-787a867ae795 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.000774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.001434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.001717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.002002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.002699 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:28.003007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.004615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.012564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbc49e630fc461a858eb04c2dd52a82', 'request_id': 'req-ebf3365d-ce8c-4b7d-9879-626cd46772f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:28.016863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1280/2546] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:47:28.021093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.031258 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3611-c090-4d55-9782-c183dcb4abbe admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.035836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:28.089272 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbc49e630fc461a858eb04c2dd52a82', 'request_id': 'req-ebf3365d-ce8c-4b7d-9879-626cd46772f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:28.090752 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.091375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.091657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.091934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.092597 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.094537 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.096274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbc49e630fc461a858eb04c2dd52a82', 'request_id': 'req-ebf3365d-ce8c-4b7d-9879-626cd46772f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:28.107900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] The token cache is being invalidate because project fdbc49e630fc461a858eb04c2dd52a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:28.108257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:28.108972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ebf3365d-ce8c-4b7d-9879-626cd46772f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:28.110567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1267/2547] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:27 2022] DELETE /identity/v3/projects/fdbc49e630fc461a858eb04c2dd52a82 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:28.116108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d97f-b843-4f33-8678-3efff8b02c28 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.116614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d97f-b843-4f33-8678-3efff8b02c28 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.117060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d97f-b843-4f33-8678-3efff8b02c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.137632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c881d97f-b843-4f33-8678-3efff8b02c28 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:28.138839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c881d97f-b843-4f33-8678-3efff8b02c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:28.148484 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1268/2548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:28.156558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.156850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.157166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.179524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:28.179903 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1281/2549] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:28.181073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:28.188230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.189934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:28.214290 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:28.243885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1269/2550] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:28.253909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.264084 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df07515-de0d-4d3c-a155-cdee51f0c8a2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.265025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04902-7580-4fed-9def-a54fad13cb2a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.269497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.270152 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.270449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.270587 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.270723 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.271058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.271343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.271753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.272249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:28.272509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.272675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.274203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.274542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.298320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1270/2551] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:47:28.302649 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.320434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c9471-1585-453e-9bf1-4bbf556436d5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.325302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.325904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.326183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.326456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.327106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.330180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.359003 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1282/2552] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:28.363100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.373696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4514c30-d20d-4be4-8729-189e0a9bed95 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.378334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.379092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.379275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.379596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.380240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.382755 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.394035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1283/2553] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:28.398557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.408737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c447b-5fe2-4126-ba9b-0ebcc76b4db5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.414513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.415170 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.415457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.415725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.416451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.418210 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.420350 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1271/2554] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:28.428559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.431771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1284/2555] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:28.435636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.446651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a40e7-3028-4182-8bdd-903f2e1199d0 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.498268 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38274346-6b6f-4789-a0cd-a13cbda4b8dd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.502975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.503595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.503889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.504168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.504834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.507326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.512247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.512923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.513240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.513516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] PATH_INFO: `/v3/projects/b9f6e0d75e204fdb98386336b9d6f416/users/1698af1e284e423dbb3c79627828c6ed/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.518099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:28.532403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f6e0d75e204fdb98386336b9d6f416, user_id=1698af1e284e423dbb3c79627828c6ed, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.535167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b216-be92-4496-937d-9f1eda24ec10 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.644929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1272/2556] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:28.647492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1285/2557] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:28 2022] PUT /identity/v3/projects/b9f6e0d75e204fdb98386336b9d6f416/users/1698af1e284e423dbb3c79627828c6ed/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:28.649072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.658957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.658957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.658957 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.660530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3897d0-ae77-43bb-8de4-af2fe066ef97 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.689478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] MFA Rules not processed for user `1698af1e284e423dbb3c79627828c6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:28.690875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:28.726382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.727003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.727284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.727683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.728435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.730164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.742076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1273/2558] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:28.746176 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.757441 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3584-bdc9-43ca-afd6-afff7d024ad1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.762327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.763031 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1286/2559] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:28.763193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.763404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.763683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.764428 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.766141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.767358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.777449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1274/2560] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:28.778212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596725d0-aeb2-4812-80c6-5ee54fe13fb1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.781414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.782779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.783382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.783660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.783933 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.784584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.786486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.791773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36bf2d-5705-4e38-a35d-e3c3f65a7d15 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.796803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:28.797483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.797760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.798034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] PATH_INFO: `/v3/projects/403863026f674e558f1fb5da31e961b9/users/27b2454188a74367a31a1bca03984f10/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.810760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:28.813672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403863026f674e558f1fb5da31e961b9, user_id=27b2454188a74367a31a1bca03984f10, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:28.816404 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188ded1-2066-49af-953f-5beb148535e8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:28.879967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1287/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:28.884051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:28.914581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1275/2562] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:28 2022] PUT /identity/v3/projects/403863026f674e558f1fb5da31e961b9/users/27b2454188a74367a31a1bca03984f10/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:28.928419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:28.928716 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:28.929036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:28.961535 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0d882-c0b1-42a1-9139-65541c55810d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:28.961765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] MFA Rules not processed for user `27b2454188a74367a31a1bca03984f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:28.962808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:29.023403 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.024183 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.024465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.024768 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.025483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.027427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.031782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1276/2563] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:29.035725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:29.036833 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.052297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00cf96-90cc-4ff8-8ccc-6fc54b5cfd9f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.058062 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.058687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.058972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.059249 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.060021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.062015 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.124703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1288/2564] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:28 2022] POST /identity/v3/users => generated 291 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:29.126720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.136197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345aef-78e8-4333-bc49-63f7cdfbec34 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.140905 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.141559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.141839 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.142137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.142806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.144384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.155329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1289/2565] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:29.160223 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.165395 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1277/2566] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:29 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:29.166224 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.172460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ffd91-616c-4cff-919f-f601e324b259 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.233650 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.234336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.234615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.234886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] PATH_INFO: `/v3/projects/771828b573f646a7a817968584666c6d/users/40e071b5a6fa487cbb36ea58d4f58311/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.236052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7dd4-db37-45ff-9887-d0154d319781 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.242642 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.243283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.243653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.243931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.244590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.246546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.252575 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771828b573f646a7a817968584666c6d, user_id=40e071b5a6fa487cbb36ea58d4f58311, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.255648 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.318819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1278/2567] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:29 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:29.323307 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.332486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1290/2568] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:29 2022] PUT /identity/v3/projects/771828b573f646a7a817968584666c6d/users/40e071b5a6fa487cbb36ea58d4f58311/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:29.334908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8afe22-8282-40ca-ac61-0249c75bddf4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.336508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.381751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:29.396037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.397678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.397678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.397933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.398464 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.400163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.402067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e15be-69b9-4342-ae72-e698a02a8270 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.407105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.407736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.408019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.408297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.409041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.410641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.410775 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1279/2569] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:29.415708 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.424203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1291/2570] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:29.426911 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d45f2-3d3f-4061-8220-f3bb33da8257 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.428799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.433093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.433093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.433388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.433640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] PATH_INFO: `/v3/projects/21a8fad9e806471898fcdaf828c4e41c/users/6ba03fce6c8a4d7d9672b4f75f8a4a28/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.438891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8de40-90b0-4dd0-b591-f434662fe616 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.444525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.445412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.445695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.445975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] PATH_INFO: `/v3/projects/771828b573f646a7a817968584666c6d/users/40e071b5a6fa487cbb36ea58d4f58311/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.447987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771828b573f646a7a817968584666c6d, user_id=40e071b5a6fa487cbb36ea58d4f58311, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.450657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6743ce4-be9a-4613-922e-4052fbd72645 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.451950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a8fad9e806471898fcdaf828c4e41c, user_id=6ba03fce6c8a4d7d9672b4f75f8a4a28, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.454644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.462201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1292/2571] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:29 2022] PUT /identity/v3/projects/771828b573f646a7a817968584666c6d/users/40e071b5a6fa487cbb36ea58d4f58311/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:29.473990 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.476770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.476770 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.504761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] MFA Rules not processed for user `40e071b5a6fa487cbb36ea58d4f58311`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:29.506637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:29.529490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1280/2572] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:29 2022] PUT /identity/v3/projects/21a8fad9e806471898fcdaf828c4e41c/users/6ba03fce6c8a4d7d9672b4f75f8a4a28/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:29.536586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.542927 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:29.578474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1293/2573] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:29.590116 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.591470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:29.603544 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04076776-a8c9-4d4d-bcd3-5d44f57558b9 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.608765 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.609443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.609724 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.610000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.610704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.612331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.623820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1281/2574] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:47:29.627674 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.637853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f44a9-904e-4611-aa09-cf585111f2be admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.642511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.643156 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.643440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.643719 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] PATH_INFO: `/v3/projects/21a8fad9e806471898fcdaf828c4e41c/users/6ba03fce6c8a4d7d9672b4f75f8a4a28/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.645722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a8fad9e806471898fcdaf828c4e41c, user_id=6ba03fce6c8a4d7d9672b4f75f8a4a28, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.649226 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227ff2-e116-4f9b-8741-3603f171d21b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.654039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13de6a-607a-4ffb-907f-464d8856a812 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.659448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1282/2575] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:29 2022] PUT /identity/v3/projects/21a8fad9e806471898fcdaf828c4e41c/users/6ba03fce6c8a4d7d9672b4f75f8a4a28/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:29.670238 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.670795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.671076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.701907 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] MFA Rules not processed for user `6ba03fce6c8a4d7d9672b4f75f8a4a28`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:29.703161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:29.705961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.706621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.706902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.707181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.776860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.778386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.780799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1283/2576] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:29.789276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1294/2577] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:47:29.792721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:29.794146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:29.817598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbd789-cc87-491e-866b-5dd3382f389f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:29.821643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:29.822278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:29.822574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:29.822839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:29.832735 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:29.834292 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:29.843941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1284/2578] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:29 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:30.002222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:30.010966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93aecd-0b3c-4d24-906e-0b0afd90b2e8 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:30.014733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:30.015339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:30.015612 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:30.015884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:30.051608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:30.076523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:30.077984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:30.080681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:30.088833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1295/2579] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:30.090431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399395a-99e0-4f27-a357-e0b97148ac6c service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:30.096553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:30.097349 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:30.097631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:30.097905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:30.111673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:30.161996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:30.164289 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958ab16-a292-46e6-a7fe-69b48e404610 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:30.174091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1285/2580] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:47:33.749230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:33.827154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef3d95-0822-4431-84f3-3468333bb0ae service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:33.832012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:33.832609 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:33.833147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:33.833301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] PATH_INFO: `/v3/users/27b2454188a74367a31a1bca03984f10` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:33.834799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b2454188a74367a31a1bca03984f10)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:33.836936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:34.117158 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b2454188a74367a31a1bca03984f10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:34.130074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b2454188a74367a31a1bca03984f10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:34.137137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b2454188a74367a31a1bca03984f10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:34.137811 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b2454188a74367a31a1bca03984f10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:34.221044 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b2454188a74367a31a1bca03984f10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:34.228549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1296/2581] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:33 2022] DELETE /identity/v3/users/27b2454188a74367a31a1bca03984f10 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:34.233188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:34.233671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:34.233778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:34.253647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:34.259970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:34.316399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1286/2582] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:34.326283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:34.327720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:34.386753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dac30-9163-4dd8-9780-8446c9d07f01 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:34.390185 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:34.390798 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:34.391079 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:34.391352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:34.398118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:34.400866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:34.402322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:34.411515 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1297/2583] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:34.821125 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:34.826497 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:34.835539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457846ba-a1bb-48c7-a93d-c10a9322f726 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:34.841169 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:34.841760 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:34.842045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:34.842327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] PATH_INFO: `/v3/projects/403863026f674e558f1fb5da31e961b9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:34.843562 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=403863026f674e558f1fb5da31e961b9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:34.845537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:35.002502 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403863026f674e558f1fb5da31e961b9', 'request_id': 'req-48146a78-8e84-45c4-877e-029d1a8d91ca'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.069526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403863026f674e558f1fb5da31e961b9', 'request_id': 'req-48146a78-8e84-45c4-877e-029d1a8d91ca'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.076640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403863026f674e558f1fb5da31e961b9', 'request_id': 'req-48146a78-8e84-45c4-877e-029d1a8d91ca'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.088218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] The token cache is being invalidate because project 403863026f674e558f1fb5da31e961b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:35.088505 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.089220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.090862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1287/2584] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:34 2022] DELETE /identity/v3/projects/403863026f674e558f1fb5da31e961b9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:35.094925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:35.105926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7dab-7bfe-4afe-abda-f75de347bad8 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:35.163723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:35.164381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:35.164657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:35.165590 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] PATH_INFO: `/v3/users/6ba03fce6c8a4d7d9672b4f75f8a4a28` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:35.175555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba03fce6c8a4d7d9672b4f75f8a4a28)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:35.175555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:35.338476 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba03fce6c8a4d7d9672b4f75f8a4a28'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.350208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba03fce6c8a4d7d9672b4f75f8a4a28'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.357397 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba03fce6c8a4d7d9672b4f75f8a4a28'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.358082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba03fce6c8a4d7d9672b4f75f8a4a28'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.423850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-918f4bb7-8899-4136-8f32-96949616b284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba03fce6c8a4d7d9672b4f75f8a4a28'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:35.430865 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1298/2585] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:35 2022] DELETE /identity/v3/users/6ba03fce6c8a4d7d9672b4f75f8a4a28 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:35.828672 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:35.830347 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:35.847423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48146a78-8e84-45c4-877e-029d1a8d91ca admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:35.884805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:35.906566 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:35.907210 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:35.907491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:35.907766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] PATH_INFO: `/v3/projects/21a8fad9e806471898fcdaf828c4e41c` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:35.908983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a8fad9e806471898fcdaf828c4e41c)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:35.910926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.032769 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a8fad9e806471898fcdaf828c4e41c', 'request_id': 'req-dbcd149b-285e-4540-9e23-84f089916afb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:36.104655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a8fad9e806471898fcdaf828c4e41c', 'request_id': 'req-dbcd149b-285e-4540-9e23-84f089916afb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:36.111609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a8fad9e806471898fcdaf828c4e41c', 'request_id': 'req-dbcd149b-285e-4540-9e23-84f089916afb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:36.122669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] The token cache is being invalidate because project 21a8fad9e806471898fcdaf828c4e41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:36.123007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:36.123652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-dbcd149b-285e-4540-9e23-84f089916afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:36.125297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1288/2586] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:35 2022] DELETE /identity/v3/projects/21a8fad9e806471898fcdaf828c4e41c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:36.130114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.130638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.130765 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.150956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:36.152175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:36.212611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1299/2587] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:47:36.220559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.220850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.221168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.240713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:36.241934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:36.250298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1289/2588] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:36.254366 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.264373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c0be-22cb-4eb6-82de-4cac8edc667a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.268915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.269547 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.269827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.270097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.270785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:36.271086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.272656 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.284696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1300/2589] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:47:36.288852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.298918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a08c3-fbc6-41ce-ba39-7217f2496b0f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.303576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.304188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.304472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.304751 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.305494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.307922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.385493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1290/2590] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:36.389763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.398342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:36.400138 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b8e5-e030-45c7-a09e-e93f697c71a2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.456524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.457157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.457438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.457710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.458369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.460254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.538154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1301/2591] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:36.542531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.553129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45bd18-bcf0-4f8e-912f-82532915013a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.558247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.558851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.559134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.559436 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.560134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.561770 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.572227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1291/2592] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:36.576069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.585878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca94f-d878-4ff1-a7e1-dbb188e65259 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.590693 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.591294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.591576 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.591853 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.592542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.594145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce616bb-f95f-4f59-ae2e-64475bf187d5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.604720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1302/2593] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:36.608812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.618983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad24ee8-fc0e-4cce-a125-375635b106d0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.623654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.624295 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.624573 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.624846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] PATH_INFO: `/v3/projects/62d127e18e46434d8bc68b4757e5ba0f/users/4c98506d9a5e48b2aef42bb07326dd42/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.640653 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d127e18e46434d8bc68b4757e5ba0f, user_id=4c98506d9a5e48b2aef42bb07326dd42, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.643309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.708948 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1292/2594] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:36 2022] PUT /identity/v3/projects/62d127e18e46434d8bc68b4757e5ba0f/users/4c98506d9a5e48b2aef42bb07326dd42/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:36.720532 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.720820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.721143 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.749300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] MFA Rules not processed for user `4c98506d9a5e48b2aef42bb07326dd42`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:36.750502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:36.813970 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1303/2595] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:36.818454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.880206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5464db1-7d3e-42a6-a0a4-db26dd08312c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:36.885614 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:36.886236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:36.886514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:36.886785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:36.887447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:36.889367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:36.899000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:36.973996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1293/2596] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:36.978773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:36.989658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541face0-1782-4d47-ba7b-9fc37374b452 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.047191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.047819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.048103 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.048383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.049090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.051091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.162275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:37.170862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1304/2597] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:36 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:37.172319 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.183149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b182e-80ec-422e-aa9c-4be3fba6505e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.187721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.188361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.188644 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.188925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.189660 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.191253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.201457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1294/2598] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:37.205497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.216354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5b6f-7134-4a98-8d13-48d39b9a224e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.221754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.222360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.222658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.222931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.223617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.225238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf54e-6ea4-487e-89d2-2a1fcff050d8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.236654 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1305/2599] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:37.241046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.251091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0c5c3-adc0-4751-850c-e2a87b465b01 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.255661 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.256313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.256596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.256875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] PATH_INFO: `/v3/projects/56ecbf5325414af3b60875c053f0208e/users/7321e074007947708d14ec71ad304e10/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.272691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ecbf5325414af3b60875c053f0208e, user_id=7321e074007947708d14ec71ad304e10, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.276130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.395587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1295/2600] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:37 2022] PUT /identity/v3/projects/56ecbf5325414af3b60875c053f0208e/users/7321e074007947708d14ec71ad304e10/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:37.406938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.407176 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.407432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.437446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] MFA Rules not processed for user `7321e074007947708d14ec71ad304e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:37.438862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:37.507446 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1306/2601] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:37.512214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.574541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913539f-f223-429a-ba86-9b7922187a70 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.579106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.579686 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.579966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.580240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.580921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.582842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.702229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1296/2602] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:37 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:37.706228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.717851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1df70-7af6-46c2-a8eb-6c07bfd23de8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.773636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.774911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.775189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.775460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.776120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.778047 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.860780 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1307/2603] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:37 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:37.864837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.875227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d25414-80be-4f85-b2f7-c8fcbce35c9a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.879906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.880519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.881098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.881098 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.881788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.883367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:37.894659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1297/2604] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:37.898925 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:37.909608 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e897a-498d-48df-ae26-8a1e527a8aa6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:37.915505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:37.915505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:37.915505 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:37.915822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] PATH_INFO: `/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a/users/8aa8332c176642328cd873d8d71b1d31/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:37.931455 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f38e7a7c8ce43a495c9eccf19fe776a, user_id=8aa8332c176642328cd873d8d71b1d31, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:37.934226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:38.004015 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1308/2605] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:37 2022] PUT /identity/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a/users/8aa8332c176642328cd873d8d71b1d31/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:38.008131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:38.018731 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f14c1-fc1a-4029-ac4a-f05dc87869cb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:38.075817 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:38.076450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:38.076733 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:38.077037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:38.077736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:38.079341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6d68f-aa15-4ccf-b2d5-e6bfa7593f51 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:38.089222 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1298/2606] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:38.093588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:38.103920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7e40b-ce07-45b7-b570-9afe69061b28 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:38.108539 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:38.109358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:38.109639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:38.109914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] PATH_INFO: `/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a/users/8aa8332c176642328cd873d8d71b1d31/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:38.111880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f38e7a7c8ce43a495c9eccf19fe776a, user_id=8aa8332c176642328cd873d8d71b1d31, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:38.114571 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:38.125585 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1309/2607] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:38 2022] PUT /identity/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a/users/8aa8332c176642328cd873d8d71b1d31/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:38.136242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:38.136530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:38.136805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:38.168212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] MFA Rules not processed for user `8aa8332c176642328cd873d8d71b1d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:38.169501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:38.182142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:38.237328 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1299/2608] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:38.250800 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:38.252451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:38.287178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:38.310661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1729f-491d-4d8b-bf6c-5e5ef71c0544 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:38.314310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:38.316295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:38.316295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:38.316295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:38.325297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:38.326754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:38.336584 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1310/2609] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:38.438446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:38.439825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:38.452101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bf704-2002-4129-9cd0-2dde41c4867f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:38.455665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:38.456270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:38.456557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:38.456831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:38.513662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:38.515089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408f721-8736-46ac-b889-d8d54eede086 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:38.523796 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1300/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:46.715421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:46.778881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1bde-bf3f-4edd-ba90-02b4b4e5c413 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:46.783231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:46.784522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:46.784522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:46.784522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] PATH_INFO: `/v3/users/4c98506d9a5e48b2aef42bb07326dd42` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:46.785538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c98506d9a5e48b2aef42bb07326dd42)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:46.787427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:46.997850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c98506d9a5e48b2aef42bb07326dd42'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.010897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c98506d9a5e48b2aef42bb07326dd42'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.017776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c98506d9a5e48b2aef42bb07326dd42'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.020135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c98506d9a5e48b2aef42bb07326dd42'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.088880 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c98506d9a5e48b2aef42bb07326dd42'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.095215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1311/2611] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:46 2022] DELETE /identity/v3/users/4c98506d9a5e48b2aef42bb07326dd42 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:47.099793 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:47.100548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:47.100548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:47.120720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:47.121954 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:47.183131 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1301/2612] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:47.192789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:47.194247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:47.253115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2927df-2146-4796-bdce-08d001d571f4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:47.256710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:47.257347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:47.257625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:47.257897 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:47.264896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:47.267384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:47.268821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:47.279367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1312/2613] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:47.683778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:47.685411 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:47.704498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e331-63e1-4f77-bfbd-b1d0120cf26f None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:47.713274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:47.714277 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:47.714834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:47.715229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] PATH_INFO: `/v3/projects/62d127e18e46434d8bc68b4757e5ba0f` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:47.716647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d127e18e46434d8bc68b4757e5ba0f)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:47.718790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:47.838548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:47.842219 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d127e18e46434d8bc68b4757e5ba0f', 'request_id': 'req-b6c94927-0824-4e79-bc68-30e660acaad0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.932797 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d127e18e46434d8bc68b4757e5ba0f', 'request_id': 'req-b6c94927-0824-4e79-bc68-30e660acaad0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.939563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d127e18e46434d8bc68b4757e5ba0f', 'request_id': 'req-b6c94927-0824-4e79-bc68-30e660acaad0'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.950595 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] The token cache is being invalidate because project 62d127e18e46434d8bc68b4757e5ba0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:47.950889 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.952662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:47.953151 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1302/2614] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:47 2022] DELETE /identity/v3/projects/62d127e18e46434d8bc68b4757e5ba0f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:47.957222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:47.967302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294fa4-fdd3-4b49-b140-fdfe7274ae39 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:48.023106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:48.023738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:48.024074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:48.024329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] PATH_INFO: `/v3/users/7321e074007947708d14ec71ad304e10` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:48.025511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7321e074007947708d14ec71ad304e10)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:48.027440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:48.265737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7321e074007947708d14ec71ad304e10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:48.278470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7321e074007947708d14ec71ad304e10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:48.285310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7321e074007947708d14ec71ad304e10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:48.285998 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7321e074007947708d14ec71ad304e10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:48.338346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7321e074007947708d14ec71ad304e10'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:48.344636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1313/2615] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:47 2022] DELETE /identity/v3/users/7321e074007947708d14ec71ad304e10 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:48.763073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:48.764582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:48.778970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c94927-0824-4e79-bc68-30e660acaad0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:48.837269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:48.837914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:48.838198 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:48.838479 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] PATH_INFO: `/v3/projects/56ecbf5325414af3b60875c053f0208e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:48.839641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ecbf5325414af3b60875c053f0208e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:48.841707 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:48.957114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ecbf5325414af3b60875c053f0208e', 'request_id': 'req-726ed8ad-52fd-46db-8932-05d99ec8c4f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.047321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ecbf5325414af3b60875c053f0208e', 'request_id': 'req-726ed8ad-52fd-46db-8932-05d99ec8c4f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.054446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ecbf5325414af3b60875c053f0208e', 'request_id': 'req-726ed8ad-52fd-46db-8932-05d99ec8c4f1'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.066421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] The token cache is being invalidate because project 56ecbf5325414af3b60875c053f0208e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:49.066777 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.067454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.068978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1303/2616] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:48 2022] DELETE /identity/v3/projects/56ecbf5325414af3b60875c053f0208e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:49.073784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:49.075233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:49.089776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a4895-a3f0-48ab-bfa9-904c4442dc26 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:49.154341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:49.154341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:49.154341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:49.154341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] PATH_INFO: `/v3/users/8aa8332c176642328cd873d8d71b1d31` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:49.154829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa8332c176642328cd873d8d71b1d31)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:49.156120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:49.176713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:49.389155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa8332c176642328cd873d8d71b1d31'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.401236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa8332c176642328cd873d8d71b1d31'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.408359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa8332c176642328cd873d8d71b1d31'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.409118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa8332c176642328cd873d8d71b1d31'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.481060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e9e91fc4-42e3-4692-8efa-fbd5fc632a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa8332c176642328cd873d8d71b1d31'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:49.487349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1314/2617] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:47:49 2022] DELETE /identity/v3/users/8aa8332c176642328cd873d8d71b1d31 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:49.912248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:49.912718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:49.927799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed8ad-52fd-46db-8932-05d99ec8c4f1 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:49.990157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:49.990808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:49.991091 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:49.991373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] PATH_INFO: `/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:49.992560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f38e7a7c8ce43a495c9eccf19fe776a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:49.994782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:49.997140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:50.170957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f38e7a7c8ce43a495c9eccf19fe776a', 'request_id': 'req-fd28d0c3-3569-4273-98cf-876fc9e1cd24'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:50.259503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f38e7a7c8ce43a495c9eccf19fe776a', 'request_id': 'req-fd28d0c3-3569-4273-98cf-876fc9e1cd24'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:50.266926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f38e7a7c8ce43a495c9eccf19fe776a', 'request_id': 'req-fd28d0c3-3569-4273-98cf-876fc9e1cd24'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:50.280804 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] The token cache is being invalidate because project 6f38e7a7c8ce43a495c9eccf19fe776a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:47:50.281128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:50.281790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-fd28d0c3-3569-4273-98cf-876fc9e1cd24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:47:50.283359 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1304/2618] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:47:49 2022] DELETE /identity/v3/projects/6f38e7a7c8ce43a495c9eccf19fe776a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:50.290220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.290833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.290833 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.311429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:50.312625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:50.371154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1315/2619] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:50.379481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.379774 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.380058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.400204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:50.401510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:50.409494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1305/2620] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:50.413870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.424328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf2872-5bc5-4543-81ad-31a5e883d531 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.428907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.429540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.429821 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.430096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.431606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:47:50.431909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.433523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.444911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1316/2621] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:47:50.450180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.459501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67181cb9-681a-48c0-953e-b5a8ab849ea5 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.464306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.464958 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.465265 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.465538 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.466189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.468077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.579287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1306/2622] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:50 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:47:50.583350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.593430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969419d-79ac-4d70-a407-baaa6de252cb admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.658032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.658672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.658952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.659222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.659960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.661829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.757312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1317/2623] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:50 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:50.762253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.772722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c979b42-4421-4929-80d0-e7f3d6c92b73 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.777699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.778313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.778598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.778879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.779581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.781214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.791713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1307/2624] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:50.795922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.806710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05e7d-f125-48a5-bcfe-79bf3161407c admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.811418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.814759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.814759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.814759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.814759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.815148 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e2e82-89f6-4d9d-9e4c-9aee193e4e8f admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.820413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:50.826432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1318/2625] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:50.830574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:50.840576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c401e-2b0c-4b50-afed-5338899b3de2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:50.845270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:50.847726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.847726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.847726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] PATH_INFO: `/v3/projects/9321b1de1c2f4f89ba51d69546c13ab9/users/fc157f58f58849d5b65723adba2b44fd/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.861645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9321b1de1c2f4f89ba51d69546c13ab9, user_id=fc157f58f58849d5b65723adba2b44fd, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:50.864302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:50.935659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1308/2626] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:50 2022] PUT /identity/v3/projects/9321b1de1c2f4f89ba51d69546c13ab9/users/fc157f58f58849d5b65723adba2b44fd/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:50.946996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:50.947326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:50.947635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:50.975506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] MFA Rules not processed for user `fc157f58f58849d5b65723adba2b44fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:50.976762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:51.044913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1319/2627] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:51.049781 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.112306 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3f33-b3d0-4aa5-9b12-b05fea57454a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.116966 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.117623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.118508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.118787 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.119565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.122171 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.237187 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1309/2628] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:47:51 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:51.241104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.252172 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312ea7f-a630-4611-863f-c682f6dabb4f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.308956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.309661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.309870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.310147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.310820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.312732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.412827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1320/2629] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:47:51 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:47:51.417901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.428575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474daf-b48c-42d8-860b-4e9d533ab0c2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.434158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.434767 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.435052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.435332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.438543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.440186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.452340 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1310/2630] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:51.456490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.467362 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286a55c-98ab-444b-baa4-5542be65e487 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.471910 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.472565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.472845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.477139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] PATH_INFO: `/v3/projects/c1e07e01e997426796cefc8f9d0692ba/users/fdd42972f33e4d7bbd0659a9734373b7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.488090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e07e01e997426796cefc8f9d0692ba, user_id=fdd42972f33e4d7bbd0659a9734373b7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.490724 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.579251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1321/2631] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:51 2022] PUT /identity/v3/projects/c1e07e01e997426796cefc8f9d0692ba/users/fdd42972f33e4d7bbd0659a9734373b7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:47:51.583166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.588771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:51.594211 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464ec1-8fdc-4d04-a5c0-9cbe4ccae605 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.654726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.655829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.656106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.656389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.657101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.658681 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd594a3-52c1-42a5-9d1f-f29dc7b795bf admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.670504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1311/2632] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:47:51.674804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.686251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394bbc6-34a3-4cfe-8cd0-bf2cfb35bc94 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.691093 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.691756 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.692038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.692317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] PATH_INFO: `/v3/projects/c1e07e01e997426796cefc8f9d0692ba/users/fdd42972f33e4d7bbd0659a9734373b7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.694346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e07e01e997426796cefc8f9d0692ba, user_id=fdd42972f33e4d7bbd0659a9734373b7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.697134 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.707828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1322/2633] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:47:51 2022] PUT /identity/v3/projects/c1e07e01e997426796cefc8f9d0692ba/users/fdd42972f33e4d7bbd0659a9734373b7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:47:51.719387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.719745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.720034 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.749722 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] MFA Rules not processed for user `fdd42972f33e4d7bbd0659a9734373b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:47:51.750851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:51.820483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1312/2634] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:47:51.832308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:51.839545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:47:51.881717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:47:51.892808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eac96-2522-403e-968f-2b00a3aabec5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:51.896593 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:51.897235 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:51.897546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:51.897825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:51.907416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:51.908851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:51.919057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1323/2635] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:47:52.213054 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:47:52.223122 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5bd66-882e-48fd-a53f-4830276dadf4 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:47:52.226794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:47:52.228072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:47:52.228351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:47:52.228625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:47:52.292784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:47:52.292784 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9bb1-912f-4557-991b-006e3d60bb18 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:47:52.299048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1313/2636] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:47:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:02.063374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=7 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:02.294437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:03.976082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:04.051058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06d937-3533-4a5e-aa5d-5d6378ad19f1 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:04.051058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:04.051058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:04.051058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:04.051438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] PATH_INFO: `/v3/users/fc157f58f58849d5b65723adba2b44fd` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:04.051438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc157f58f58849d5b65723adba2b44fd)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:04.051963 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:04.238299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc157f58f58849d5b65723adba2b44fd'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:04.250871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc157f58f58849d5b65723adba2b44fd'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:04.257696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc157f58f58849d5b65723adba2b44fd'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:04.258378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc157f58f58849d5b65723adba2b44fd'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:04.343500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc157f58f58849d5b65723adba2b44fd'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:04.351655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1324/2637] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:03 2022] DELETE /identity/v3/users/fc157f58f58849d5b65723adba2b44fd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:04.355524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:04.356007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:04.356120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:04.375638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:04.376845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:04.436204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1314/2638] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:04.446069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:04.448111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:04.570102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e28d99-7714-4a14-9b7f-7c0a12d0140b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:04.570102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:04.570102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:04.570102 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:04.572479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:04.572479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:04.572479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:04.572479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1325/2639] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:48:04.982127 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:04.983776 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:04.999088 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35165ef-f0b5-45b9-a175-25dd94f932cc None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:05.003932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:05.004657 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:05.004972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:05.005284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] PATH_INFO: `/v3/projects/9321b1de1c2f4f89ba51d69546c13ab9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:05.006456 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9321b1de1c2f4f89ba51d69546c13ab9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:05.008408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:05.095470 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9321b1de1c2f4f89ba51d69546c13ab9', 'request_id': 'req-de53e604-934f-48a3-9c53-9a7d11c479b9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.172316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9321b1de1c2f4f89ba51d69546c13ab9', 'request_id': 'req-de53e604-934f-48a3-9c53-9a7d11c479b9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.179140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9321b1de1c2f4f89ba51d69546c13ab9', 'request_id': 'req-de53e604-934f-48a3-9c53-9a7d11c479b9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.190157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] The token cache is being invalidate because project 9321b1de1c2f4f89ba51d69546c13ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:05.190449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.191078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.192617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1315/2640] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:04 2022] DELETE /identity/v3/projects/9321b1de1c2f4f89ba51d69546c13ab9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:05.196586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:05.206209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd607cd-b52d-40b9-9d62-478b4b7ca67b service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:05.224378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:05.263099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:05.263736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:05.264018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:05.264296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] PATH_INFO: `/v3/users/fdd42972f33e4d7bbd0659a9734373b7` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:05.265482 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd42972f33e4d7bbd0659a9734373b7)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:05.274399 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:05.447804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd42972f33e4d7bbd0659a9734373b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.459955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd42972f33e4d7bbd0659a9734373b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.467553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd42972f33e4d7bbd0659a9734373b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.468252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd42972f33e4d7bbd0659a9734373b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.560703 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-4e4f6090-0f24-4050-ba8c-c18893355d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd42972f33e4d7bbd0659a9734373b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:05.566832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1326/2641] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:05 2022] DELETE /identity/v3/users/fdd42972f33e4d7bbd0659a9734373b7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:05.999622 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.001491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:06.015100 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53e604-934f-48a3-9c53-9a7d11c479b9 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.078228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.078874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.079189 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.079467 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] PATH_INFO: `/v3/projects/c1e07e01e997426796cefc8f9d0692ba` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.080654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e07e01e997426796cefc8f9d0692ba)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.082626 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.158114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:06.177809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e07e01e997426796cefc8f9d0692ba', 'request_id': 'req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:06.279425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e07e01e997426796cefc8f9d0692ba', 'request_id': 'req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:06.283706 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e07e01e997426796cefc8f9d0692ba', 'request_id': 'req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:06.295490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] The token cache is being invalidate because project c1e07e01e997426796cefc8f9d0692ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:06.295782 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:06.296518 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0704ac09-3a4e-4e0c-ba49-b7ff4a0373b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:06.298154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1316/2642] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:05 2022] DELETE /identity/v3/projects/c1e07e01e997426796cefc8f9d0692ba => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:06.303331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.303818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.303926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.325113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:06.326312 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:06.385832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1327/2643] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:06.394033 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.394322 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.394601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.416829 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:06.418064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:06.427824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1317/2644] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:06.432160 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.441988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c286f10-8035-4fcf-a99d-993b460a6325 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.447556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.448180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.448465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.448738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.449480 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:06.449787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.451354 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.463687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1328/2645] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:48:06.468217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.477414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff740c-468b-46e1-99cd-cd46d92f85ad None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.483180 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.485801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.485801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.485801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.485801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.486942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.588617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1318/2646] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:06 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:06.592758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.602729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c5a5-aa12-4914-8cfa-783810faa243 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.628613 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:06.660301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.660919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.661343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.661668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.662329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.664229 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.758201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1329/2647] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:06 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:06.763229 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.773541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2ed9a-8975-457f-8193-c2706951fbb5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.779600 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.780524 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.780806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.781118 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.786261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.786261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.794019 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1319/2648] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:06.798361 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.808776 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa79b71-44ce-4d53-821c-507f2d0c01d9 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.813807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.814417 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.814812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.815090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.815781 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.817432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ca2cc-d875-4862-9163-cae000e04a59 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.828961 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1330/2649] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:06.832924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:06.843757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56a840-888c-4584-873c-b23b5ade8c5c admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:06.848421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:06.849099 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.849383 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.849693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] PATH_INFO: `/v3/projects/7190d85b7f07434b992bd792dfc9ff87/users/bd43b25862e2404bb73dca97e4262ac9/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.865124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7190d85b7f07434b992bd792dfc9ff87, user_id=bd43b25862e2404bb73dca97e4262ac9, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:06.867780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:06.934650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1320/2650] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:06 2022] PUT /identity/v3/projects/7190d85b7f07434b992bd792dfc9ff87/users/bd43b25862e2404bb73dca97e4262ac9/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:06.946414 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:06.946709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:06.946988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:06.977327 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] MFA Rules not processed for user `bd43b25862e2404bb73dca97e4262ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:06.981276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:07.054845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1331/2651] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:07.066990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:07.068790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:07.129853 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6eb00-5cd5-40b1-a5f6-976333c086a5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:07.133586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:07.134885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:07.135179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:07.135550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:07.144712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:07.146179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09330423-85d6-4c28-959e-359c1e6681a7 service glance] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:07.156392 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1321/2652] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:17.051818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:17.152120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:35.132940 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:35.134553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:35.199731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29abfd-db1f-4786-854a-05492cc70204 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:35.204114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:35.204747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:35.205049 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:35.205949 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] PATH_INFO: `/v3/users/f3424f871a224e8d92733a9556ce8284` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:35.207089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3424f871a224e8d92733a9556ce8284)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:35.209080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:35.445381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3424f871a224e8d92733a9556ce8284'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:35.460266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3424f871a224e8d92733a9556ce8284'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:35.467591 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3424f871a224e8d92733a9556ce8284'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:35.468334 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3424f871a224e8d92733a9556ce8284'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:35.527907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3424f871a224e8d92733a9556ce8284'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:35.534019 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1332/2653] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:35 2022] DELETE /identity/v3/users/f3424f871a224e8d92733a9556ce8284 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:35.540468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:35.540468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:35.540468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:35.558540 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:35.559718 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:35.621243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1322/2654] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:35.631518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:35.632894 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:35.700568 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c9de8-7c8d-4994-a2cf-f4ee89f71666 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:35.704203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:35.704838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:35.705169 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:35.705553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:35.714502 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:35.715926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012e807-9984-408e-826d-ecc2556f0c7a service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:35.725330 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1333/2655] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:36.176885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.178407 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:36.190983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd7fb6-378f-4b8c-844c-de9784768eb7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.195805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.196435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.196711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.196983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] PATH_INFO: `/v3/projects/fb55231013b4448ca78b749a09c5c3ff` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.201121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb55231013b4448ca78b749a09c5c3ff)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.201121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.311756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb55231013b4448ca78b749a09c5c3ff', 'request_id': 'req-1a5ea04d-5c10-4f09-b6e5-c178797f437c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:36.397007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb55231013b4448ca78b749a09c5c3ff', 'request_id': 'req-1a5ea04d-5c10-4f09-b6e5-c178797f437c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:36.403641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb55231013b4448ca78b749a09c5c3ff', 'request_id': 'req-1a5ea04d-5c10-4f09-b6e5-c178797f437c'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:36.414565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] The token cache is being invalidate because project fb55231013b4448ca78b749a09c5c3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:36.414848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:36.415474 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1a5ea04d-5c10-4f09-b6e5-c178797f437c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:36.416984 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1323/2656] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:36 2022] DELETE /identity/v3/projects/fb55231013b4448ca78b749a09c5c3ff => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:36.422231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.422742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.422851 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.442081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:36.443216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:36.460075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:36.501844 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1334/2657] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:36.509799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.510083 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.510378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.530720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:36.532710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:36.540900 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1324/2658] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:36.545372 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.555423 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76c189-23b7-40ed-82bf-45605e6ac965 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.560194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.567291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.575743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1335/2659] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:48:36.579890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.589697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f16d0-8551-4d43-be65-78821aeaad3d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.594378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.594983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.595264 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.595543 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.596819 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.598756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.683625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1325/2660] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:36 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:36.687697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.698511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef878746-4909-4292-abb0-d5c6bea87013 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.756083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.756705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.757008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.757300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.757976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.759917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.862891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1336/2661] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:36 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:48:36.867128 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.878309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b088a45-386c-4243-822a-37976c3ed5b3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.882870 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.883677 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.883930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.884037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.884788 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.886408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.897400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1326/2662] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:36.901259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.911425 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d7f4-b2ed-4789-949c-f39edc2df799 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.916289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.916904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.917286 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.917564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.918264 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.919850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687cb6-97f6-4b32-bed1-45d1b11c8dfb admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:36.930978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1337/2663] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:36.935052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:36.945872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2c776-befd-48d4-9698-6a96fb86d895 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:36.950577 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:36.951177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:36.951460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:36.951739 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] PATH_INFO: `/v3/projects/8ec72d9351e942258cf5a668c7fe34a4/users/62ae350eca5f463d98f59ae338a30600/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:36.966916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec72d9351e942258cf5a668c7fe34a4, user_id=62ae350eca5f463d98f59ae338a30600, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:36.969669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:37.069758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1327/2664] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:36 2022] PUT /identity/v3/projects/8ec72d9351e942258cf5a668c7fe34a4/users/62ae350eca5f463d98f59ae338a30600/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:37.080013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:37.080638 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:37.080997 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:37.109016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] MFA Rules not processed for user `62ae350eca5f463d98f59ae338a30600`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:37.109950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:37.183427 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1338/2665] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:37.195353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:37.197123 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:37.219008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:37.255046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5069323-0a15-4c8d-a7c7-9fd774404eb4 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:37.258847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:37.259712 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:37.260058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:37.260508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:37.269351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:37.271922 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc3cec-7d36-4740-8d3f-f44dc6070c0c service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:37.279463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1328/2666] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:45.606796 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:45.626544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:45.670810 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4ad5d-cb31-446d-b0aa-1624f073df5f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:45.675701 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:45.676382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:45.676662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:45.676938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] PATH_INFO: `/v3/users/62ae350eca5f463d98f59ae338a30600` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:45.678198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ae350eca5f463d98f59ae338a30600)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:45.680248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:45.870854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ae350eca5f463d98f59ae338a30600'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:45.883135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ae350eca5f463d98f59ae338a30600'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:45.889759 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ae350eca5f463d98f59ae338a30600'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:45.890431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ae350eca5f463d98f59ae338a30600'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:45.979563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ae350eca5f463d98f59ae338a30600'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:45.986492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1339/2667] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:45 2022] DELETE /identity/v3/users/62ae350eca5f463d98f59ae338a30600 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:45.991592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:45.992059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:45.992161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:46.013115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:46.017924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:46.074196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1329/2668] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:46.083960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:46.085378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:46.143137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb595-5cd8-480c-9761-0856f622d140 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:46.146996 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:46.147603 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:46.147876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:46.148147 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:46.157592 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:46.159089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd1b0-a5ce-47c0-9342-045f8233a694 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:46.169470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1340/2669] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:46.617972 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:46.632934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:46.634192 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af272f-705b-4267-a034-f1dcabf4486d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:46.647006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:46.647863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:46.648244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:46.648799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] PATH_INFO: `/v3/projects/8ec72d9351e942258cf5a668c7fe34a4` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:46.650460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec72d9351e942258cf5a668c7fe34a4)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:46.652959 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:46.797379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec72d9351e942258cf5a668c7fe34a4', 'request_id': 'req-1525ec2f-ced8-4431-aca2-9b5fff127791'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:46.899822 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec72d9351e942258cf5a668c7fe34a4', 'request_id': 'req-1525ec2f-ced8-4431-aca2-9b5fff127791'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:46.909159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec72d9351e942258cf5a668c7fe34a4', 'request_id': 'req-1525ec2f-ced8-4431-aca2-9b5fff127791'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:46.918504 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] The token cache is being invalidate because project 8ec72d9351e942258cf5a668c7fe34a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:46.918792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:46.919472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-1525ec2f-ced8-4431-aca2-9b5fff127791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:46.921331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1330/2670] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:46 2022] DELETE /identity/v3/projects/8ec72d9351e942258cf5a668c7fe34a4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:46.926867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:46.927563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:46.927563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:46.951321 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:46.952729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:46.990541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:47.024091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1341/2671] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:47.033074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.033571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.033679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.054987 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:47.056227 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:47.064278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1331/2672] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:47.069215 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.078921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7c03-4db8-4f56-bed5-24c7ab85ab5b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.083491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.084125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.084406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.084680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.085464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:47.085771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.087352 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.099440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1342/2673] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:48:47.103897 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.113343 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323cb74-dfad-46a6-99bf-bf46f5c46ddd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.118839 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.119478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.119755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.120029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.120688 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.122609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.207942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1332/2674] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:47 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:47.212052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.222012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5941433-ebc6-4ff7-86c5-d61e346dafba admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.281715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.282382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.282666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.282944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.283619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.286872 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.380529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1343/2675] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:47 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:47.384679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.396299 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0821f63-1657-4fba-824f-439064be91e8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.400918 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.401639 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.401934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.402363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.403108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.404691 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.415218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1333/2676] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:47.419322 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.430507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec77919-a152-444f-bcd0-f4124319332b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.435251 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.436089 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.436369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.436645 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.437390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.439017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef56d1-dcbb-43b0-961c-77013546add0 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.575287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1344/2677] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:47.579564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.589581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a01f-d20c-4f78-9e75-0c72a8a84349 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.594597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.595286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.595564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.595840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] PATH_INFO: `/v3/projects/99b4f140f08e4120a1503c7ecc3e735a/users/36bf876a40df413b90f79fdef7c98726/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.598318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:47.611503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b4f140f08e4120a1503c7ecc3e735a, user_id=36bf876a40df413b90f79fdef7c98726, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.615120 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.700666 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1334/2678] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:47 2022] PUT /identity/v3/projects/99b4f140f08e4120a1503c7ecc3e735a/users/36bf876a40df413b90f79fdef7c98726/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:47.711512 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.711801 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.712080 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.739310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] MFA Rules not processed for user `36bf876a40df413b90f79fdef7c98726`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:47.740516 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:47.806973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1345/2679] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:47.819149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.821430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:47.881060 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c18daa-373a-4602-b3c2-4c47197ab9e5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:47.884634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86fe5edd283422cb3d5650845927693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:47.885288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:47.885571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:47.885846 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:47.894316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:47.895790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9735e0a-8eb5-49fe-8dbe-2df5a884cb19 service glance] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:47.905827 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1335/2680] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:47.917944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:47.919458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:47.964523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:47.998804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db50d-9398-4ae0-9e56-da96d420f8fa None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:48.000289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:48.000935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:48.001261 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:48.001542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] PATH_INFO: `/v3/users/bd43b25862e2404bb73dca97e4262ac9` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:48.002736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd43b25862e2404bb73dca97e4262ac9)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:48.004722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:48.234942 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd43b25862e2404bb73dca97e4262ac9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:48.250435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd43b25862e2404bb73dca97e4262ac9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:48.259663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd43b25862e2404bb73dca97e4262ac9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:48.260380 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd43b25862e2404bb73dca97e4262ac9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:48.348789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd43b25862e2404bb73dca97e4262ac9'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:48.355389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1346/2681] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:47 2022] DELETE /identity/v3/users/bd43b25862e2404bb73dca97e4262ac9 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:48.360164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:48.360645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:48.360745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:48.380425 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:48.381745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:48.447705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1336/2682] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:48.457523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:48.459028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:48.519890 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab4ae6-a0eb-4ca2-98b1-8da3776b5502 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:48.523818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:48.524460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:48.524735 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:48.525032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:48.533891 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:48.535302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:48.545349 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1347/2683] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:48.779472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:48.780914 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:48.795736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dea53-d7e6-4668-8f99-8f6378baf265 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:48.800218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:48.800641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:48.800965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:48.801234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] PATH_INFO: `/v3/users/1698af1e284e423dbb3c79627828c6ed` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:48.802530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1698af1e284e423dbb3c79627828c6ed)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:48.808052 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.017516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1698af1e284e423dbb3c79627828c6ed'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.029547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1698af1e284e423dbb3c79627828c6ed'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.042333 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.043795 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.060212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80510e1d-3ce5-4a5a-8cca-6a844815454e service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.065187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.065954 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.066230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.066554 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] PATH_INFO: `/v3/projects/7190d85b7f07434b992bd792dfc9ff87` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.067790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7190d85b7f07434b992bd792dfc9ff87)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.070500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.097515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1698af1e284e423dbb3c79627828c6ed'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.105495 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1698af1e284e423dbb3c79627828c6ed'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.106146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-653c525b-c537-492b-ae3c-0af8f67e8347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1698af1e284e423dbb3c79627828c6ed'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.107534 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:49.112738 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1337/2684] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:48 2022] DELETE /identity/v3/users/1698af1e284e423dbb3c79627828c6ed => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:49.117579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.117868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.118184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.137362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:49.138565 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.195556 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7190d85b7f07434b992bd792dfc9ff87', 'request_id': 'req-637b9968-d5a0-430d-98b3-9a0c69121df1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.198726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:49.204458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7190d85b7f07434b992bd792dfc9ff87', 'request_id': 'req-637b9968-d5a0-430d-98b3-9a0c69121df1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.204597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1338/2685] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:49.210638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.211992 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.270262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c6a46-01ad-4e42-bb0c-8bfaeafd5dbb None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.276908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7190d85b7f07434b992bd792dfc9ff87', 'request_id': 'req-637b9968-d5a0-430d-98b3-9a0c69121df1'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.288974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] The token cache is being invalidate because project 7190d85b7f07434b992bd792dfc9ff87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:49.292904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.292904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-637b9968-d5a0-430d-98b3-9a0c69121df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:49.292904 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1348/2686] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:49 2022] DELETE /identity/v3/projects/7190d85b7f07434b992bd792dfc9ff87 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:48:49.298090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1755ad-252d-4ce5-9da6-81f53f1b2ad5 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.298620 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1755ad-252d-4ce5-9da6-81f53f1b2ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.299619 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1755ad-252d-4ce5-9da6-81f53f1b2ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.323215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.323851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.324132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.324403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.324514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fe1755ad-252d-4ce5-9da6-81f53f1b2ad5 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:49.326361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.326533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fe1755ad-252d-4ce5-9da6-81f53f1b2ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.386731 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1349/2687] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:49.392415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.394321 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.394549 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.394835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.395120 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.413191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1339/2688] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:48:49.417757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:49.419130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.428566 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1350/2689] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:49.432890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.439632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=100 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:49.444324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a491a72-530e-4a24-839b-4da5e49db585 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.450246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.450874 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.453439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.453439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.453439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:49.453439 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.454193 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.468376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1340/2690] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:48:49.472747 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.482493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915540d-6429-4063-a5cf-3efe85916ea3 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.489930 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.490734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.491033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.491282 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.492024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.493964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.627887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1351/2691] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:49 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:48:49.629709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.652448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cb82c-c9eb-444d-88a3-23d9bb76690b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.713457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.714101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.717132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.717132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.717132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.717415 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.821768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1341/2692] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:49 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:49.823150 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.833799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca87e34-4492-45bf-b27a-0617adde52f1 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.850472 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.853409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.855201 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:49.872381 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.872758 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.873157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.873884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.875543 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.885269 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9ab84-5bde-4d25-8e83-62a477c215c0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.890821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.890821 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.891057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1352/2693] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:49.891160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.891160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] PATH_INFO: `/v3/projects/b9f6e0d75e204fdb98386336b9d6f416` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.891605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f6e0d75e204fdb98386336b9d6f416)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.893559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.894838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.906702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158dd15-2aa2-422d-bf17-49e418c63691 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.913558 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.914156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.914441 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.919203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.919203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.919203 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:49.928803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1353/2694] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:48:49.933016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:49.942688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732aa84-30b2-4b61-bd9a-a7ad5489df80 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:49.947983 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:49.948726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:49.949018 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:49.949443 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] PATH_INFO: `/v3/projects/ffcc4bd6d9674713a6cd0aa930589fa8/users/3e58f319ab1844a2b923e07f2e4b07c9/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:49.969495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcc4bd6d9674713a6cd0aa930589fa8, user_id=3e58f319ab1844a2b923e07f2e4b07c9, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:49.972809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06100b91-0ee4-4837-b83b-8cfac65935f1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:50.054162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1354/2695] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:49 2022] PUT /identity/v3/projects/ffcc4bd6d9674713a6cd0aa930589fa8/users/3e58f319ab1844a2b923e07f2e4b07c9/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:48:50.060285 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f6e0d75e204fdb98386336b9d6f416', 'request_id': 'req-eb3081b5-dd62-428f-93f7-d897e6eb56f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.064474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:50.064794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:50.065155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:50.094958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] MFA Rules not processed for user `3e58f319ab1844a2b923e07f2e4b07c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:50.096062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:50.114472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f6e0d75e204fdb98386336b9d6f416', 'request_id': 'req-eb3081b5-dd62-428f-93f7-d897e6eb56f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.121986 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f6e0d75e204fdb98386336b9d6f416', 'request_id': 'req-eb3081b5-dd62-428f-93f7-d897e6eb56f8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.133858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] The token cache is being invalidate because project b9f6e0d75e204fdb98386336b9d6f416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:50.134145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.134813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.137036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1342/2696] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:49 2022] DELETE /identity/v3/projects/b9f6e0d75e204fdb98386336b9d6f416 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:50.140879 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:50.142288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:50.159091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:50.168657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1355/2697] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:48:50.180067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:50.181514 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:50.216885 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3081b5-dd62-428f-93f7-d897e6eb56f8 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:50.224919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:50.242917 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee10c93-f10a-4f79-90df-c458af21effb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:50.246388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:50.247041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:50.247319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:50.247589 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:50.257536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:50.259070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:50.270496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1356/2698] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:48:50.402188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:50.402842 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:50.403126 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:50.403408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] PATH_INFO: `/v3/users/40e071b5a6fa487cbb36ea58d4f58311` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:50.404747 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e071b5a6fa487cbb36ea58d4f58311)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:50.406627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:50.575510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e071b5a6fa487cbb36ea58d4f58311'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.587471 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e071b5a6fa487cbb36ea58d4f58311'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.694772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e071b5a6fa487cbb36ea58d4f58311'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.702468 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e071b5a6fa487cbb36ea58d4f58311'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.703136 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-d55d9408-a452-4ab1-b83a-079392c8ae18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e071b5a6fa487cbb36ea58d4f58311'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:50.709278 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1343/2699] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:50 2022] DELETE /identity/v3/users/40e071b5a6fa487cbb36ea58d4f58311 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:48:51.129469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.142950 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937c9e-0f0f-43b7-bc7f-f18ada6e3ac1 service cinder] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.222104 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.222754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.223037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.223311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] PATH_INFO: `/v3/projects/771828b573f646a7a817968584666c6d` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.225115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771828b573f646a7a817968584666c6d)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.227178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.350660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771828b573f646a7a817968584666c6d', 'request_id': 'req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:51.358350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771828b573f646a7a817968584666c6d', 'request_id': 'req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:51.446323 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771828b573f646a7a817968584666c6d', 'request_id': 'req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:51.458908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] The token cache is being invalidate because project 771828b573f646a7a817968584666c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:51.459300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:51.460038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fa16ebc2-94fb-4986-9afe-2d1cefc72f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:51.461595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1357/2700] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:51 2022] DELETE /identity/v3/projects/771828b573f646a7a817968584666c6d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:51.467800 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.468315 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.468809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.492561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:51.498107 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:51.557811 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1344/2701] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:51.567202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.567567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.567936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.589106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:51.590722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:51.598932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1358/2702] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:51.603623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.613357 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3276e-5589-4635-9028-0b319b50845d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.618480 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.619110 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.619385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.619655 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.620339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:51.620636 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.623539 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.636664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1345/2703] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:48:51.641193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.650859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecdab6-866c-4d53-bc53-cc1b1bcefc1a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.655557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.656125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.656384 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.656659 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.657367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.659233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.746709 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:51.753636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1359/2704] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:51 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:51.757450 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.767363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af6a4-3008-47c9-8860-fa64f1720e29 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.780329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:51.825421 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.826038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.826318 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.826607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.827255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.829225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.917875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1346/2705] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:51 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:51.922771 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.936253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957dbb0-1c48-4fa5-a26c-ce679d3a76ae admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.941062 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.941681 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.941958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.942233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.942922 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.945211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.955794 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1360/2706] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:51.960656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:51.970933 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd08ec-d033-4de7-b289-53d060bc8125 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:51.977352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:51.983183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:51.983183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:51.983183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:51.983183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:51.983183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8075f8-4846-4855-93ba-edfe82b1cc3d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:51.993795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1347/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:48:52.008625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.009555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37924e-6417-465e-bf7d-ea2cabd03cc5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.014405 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.015142 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.015486 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.015761 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] PATH_INFO: `/v3/projects/bac941ee247a493daab54f2aee871ebb/users/5ff8ff414d374a3dbc7c632d56e34d5e/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.043328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac941ee247a493daab54f2aee871ebb, user_id=5ff8ff414d374a3dbc7c632d56e34d5e, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.046607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.135145 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1361/2708] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:51 2022] PUT /identity/v3/projects/bac941ee247a493daab54f2aee871ebb/users/5ff8ff414d374a3dbc7c632d56e34d5e/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:52.145858 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.146181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.146440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.175892 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] MFA Rules not processed for user `5ff8ff414d374a3dbc7c632d56e34d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:52.177036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:52.246756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1348/2709] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:52.251410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.315975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a0b7-3dc5-40dc-898d-d78af35079f6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.320869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.321529 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.321808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.322081 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.322775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.324657 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.437429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1362/2710] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:52 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:52.441369 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.452785 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f60eb-08e3-4307-992f-b21a21efb6b5 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.513209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.513993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.514393 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.514823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.515697 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.517923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.597373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1349/2711] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:52 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:52.602552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.612646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af9c5-7a55-4aa7-a694-ae2b4e503288 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.618002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.618605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.618908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.619198 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.619867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.621481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.632690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1363/2712] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:52.637065 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.647241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd69fb-7693-4bae-97a0-dea7b01882aa admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.651894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.652547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.652830 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.653145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] PATH_INFO: `/v3/projects/95f18769f8f54cb3bbf173de668adcdd/users/f2535dd1a0ca48d98ad8680fccac5cb7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.667447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:52.670873 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f18769f8f54cb3bbf173de668adcdd, user_id=f2535dd1a0ca48d98ad8680fccac5cb7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.673574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.752121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1350/2713] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:52 2022] PUT /identity/v3/projects/95f18769f8f54cb3bbf173de668adcdd/users/f2535dd1a0ca48d98ad8680fccac5cb7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:52.756129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.766193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e8b1-1a5b-4b39-9563-978960e2320d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.824422 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.825066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.825339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.825616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.826311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.827901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bf58-b347-45e8-8755-d482f147a0e1 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.838696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1364/2714] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:52.843072 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:52.854021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc707f6c-a720-4816-9a13-7b7fca97db5f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:52.858772 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:52.859457 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.859734 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.860007 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] PATH_INFO: `/v3/projects/95f18769f8f54cb3bbf173de668adcdd/users/f2535dd1a0ca48d98ad8680fccac5cb7/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.862042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f18769f8f54cb3bbf173de668adcdd, user_id=f2535dd1a0ca48d98ad8680fccac5cb7, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:52.864654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:52.875235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1351/2715] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:52 2022] PUT /identity/v3/projects/95f18769f8f54cb3bbf173de668adcdd/users/f2535dd1a0ca48d98ad8680fccac5cb7/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:52.886449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:52.886822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:52.887106 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:52.917096 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] MFA Rules not processed for user `f2535dd1a0ca48d98ad8680fccac5cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:52.918316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:52.946359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:52.989666 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1365/2716] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:53.002794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:53.004437 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:53.072546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad830-9c2d-4dd4-86f4-07d22e870bcb admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:53.076482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:53.077244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:53.077409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:53.077680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:53.086843 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:53.088260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:53.098092 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1352/2717] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:53.302219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:53.312276 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09070-4772-4b27-a81f-2a0814b3b621 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:53.316092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:53.316722 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:53.317013 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:53.317304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:53.380279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:53.380279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f49b5-1fc7-4714-928e-ac2f4a073d63 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:53.389924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1366/2718] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:55.465353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:55.466812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:55.555236 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260b10-48e9-4258-8895-30f9ac2da99f service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:55.560386 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:55.561190 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:55.561582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:55.561947 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] PATH_INFO: `/v3/users/3e58f319ab1844a2b923e07f2e4b07c9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:55.563157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e58f319ab1844a2b923e07f2e4b07c9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:55.565164 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:55.844366 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e58f319ab1844a2b923e07f2e4b07c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:55.856932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e58f319ab1844a2b923e07f2e4b07c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:55.939529 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e58f319ab1844a2b923e07f2e4b07c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:55.947652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e58f319ab1844a2b923e07f2e4b07c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:55.948324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e58f319ab1844a2b923e07f2e4b07c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:55.950575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:55.954939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1353/2719] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:55 2022] DELETE /identity/v3/users/3e58f319ab1844a2b923e07f2e4b07c9 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:55.959624 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:55.960194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:55.960194 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:55.979680 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:55.980876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:56.043197 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1367/2720] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:56.053795 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:56.055511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:56.116634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc99c-3950-4c37-af92-8a0ef437bc03 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:56.120301 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:56.120934 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.121225 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.121498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.130493 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:56.132324 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fb295-956d-46b1-aa81-4cae2170045b service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:56.143599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1354/2721] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:56.556461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:56.558250 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:56.571009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23f4b8-c5d2-481f-a43a-4d0a5e5e9034 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:56.581937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:56.581937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.581937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.581937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] PATH_INFO: `/v3/projects/ffcc4bd6d9674713a6cd0aa930589fa8` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.582275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffcc4bd6d9674713a6cd0aa930589fa8)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:56.583140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:56.689653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffcc4bd6d9674713a6cd0aa930589fa8', 'request_id': 'req-60565062-95b1-4f15-a6a2-15f2e5ccd286'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:56.696534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffcc4bd6d9674713a6cd0aa930589fa8', 'request_id': 'req-60565062-95b1-4f15-a6a2-15f2e5ccd286'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:56.698988 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:56.782921 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffcc4bd6d9674713a6cd0aa930589fa8', 'request_id': 'req-60565062-95b1-4f15-a6a2-15f2e5ccd286'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:56.790101 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] The token cache is being invalidate because project ffcc4bd6d9674713a6cd0aa930589fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:56.790388 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:56.791039 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-60565062-95b1-4f15-a6a2-15f2e5ccd286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:56.792544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1368/2722] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:56 2022] DELETE /identity/v3/projects/ffcc4bd6d9674713a6cd0aa930589fa8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:56.798893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.799408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.799514 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.821482 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:56.822694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:56.884422 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1355/2723] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:56.892832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.893182 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.893456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.913004 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:56.916274 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:56.922639 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1369/2724] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:56.927064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:56.937481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa35bd-945c-488d-8f62-28b2ad6cd484 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:56.942158 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:56.942709 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.942990 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.943267 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.944531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:56.944923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:56.946628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:56.960241 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1356/2725] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:48:56.964634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:56.973845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7db090-34b2-4f0b-ba84-d443c3322674 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:56.978859 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:56.979506 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:56.979809 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:56.980091 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:56.980736 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:56.982708 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.107820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1370/2726] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:56 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:57.111806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.121893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf3404-6351-4610-986c-c062409f5804 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.180061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.180680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.180962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.181263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.181930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.183844 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.292333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1357/2727] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:57 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:57.296663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.307374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0f12-7a2b-4115-8290-fdb405283445 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.311663 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.312259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.312536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.312804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.313527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.315087 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.325827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1371/2728] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:57.329814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.343274 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30ada-cd8a-47f3-ad95-a321a9729708 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.348242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.348905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.349218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.349494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.350308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.352336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dfdeb-cbff-4e98-8032-25ad34235de8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.363401 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1358/2729] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:48:57.367792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.377616 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a180ec5-8829-4fef-8ecd-3799487dbcfd admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.383108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.383804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.384082 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.384377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] PATH_INFO: `/v3/projects/b0c00e8358be425892d62a6e12be20cb/users/c16cdf84ba7e41518120e8de3c95e851/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.400831 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c00e8358be425892d62a6e12be20cb, user_id=c16cdf84ba7e41518120e8de3c95e851, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.403510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.498367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1372/2730] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:57 2022] PUT /identity/v3/projects/b0c00e8358be425892d62a6e12be20cb/users/c16cdf84ba7e41518120e8de3c95e851/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:57.509478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.509766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.511583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.533715 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:57.542217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] MFA Rules not processed for user `c16cdf84ba7e41518120e8de3c95e851`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:57.543491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:57.608280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1359/2731] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:57.620109 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.621819 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:57.686669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de255a-022c-423a-a4f2-be89b4b6a318 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.690483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.691099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.691373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.691643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.699884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.701612 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7ed12-a60d-46d4-b275-a1b874b48328 service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.711072 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1373/2732] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:57.736979 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:57.801741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4009a-4d90-4881-9c61-04c8d238329d None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:57.806951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:57.807625 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:57.807893 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:57.808173 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] PATH_INFO: `/v3/users/c16cdf84ba7e41518120e8de3c95e851` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:57.809486 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16cdf84ba7e41518120e8de3c95e851)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:57.811424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:57.988146 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16cdf84ba7e41518120e8de3c95e851'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.000043 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16cdf84ba7e41518120e8de3c95e851'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.082831 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16cdf84ba7e41518120e8de3c95e851'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.091977 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16cdf84ba7e41518120e8de3c95e851'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.092640 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16cdf84ba7e41518120e8de3c95e851'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.098752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1360/2733] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:48:57 2022] DELETE /identity/v3/users/c16cdf84ba7e41518120e8de3c95e851 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:58.103642 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:58.104156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:58.104273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:58.125494 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:58.126704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:58.192694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1374/2734] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:58.204351 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:58.205836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:58.229188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:58.268837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e17cc-aa77-45be-9978-46609bf436cd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:58.273656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:58.274282 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:58.274537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:58.274840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:58.284314 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:58.285790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd36208-8b89-4b6d-b832-02fc7bf472de service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:58.296215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1361/2735] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:48:58.701492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:58.702987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:58.715359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24693c-ebc2-45a4-93be-6fa2fa9dfd60 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:58.720637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:58.721371 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:58.721643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:58.721915 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] PATH_INFO: `/v3/projects/b0c00e8358be425892d62a6e12be20cb` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:58.723084 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0c00e8358be425892d62a6e12be20cb)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:58.725649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:58.744785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:58.831121 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0c00e8358be425892d62a6e12be20cb', 'request_id': 'req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.837704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0c00e8358be425892d62a6e12be20cb', 'request_id': 'req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.910582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0c00e8358be425892d62a6e12be20cb', 'request_id': 'req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.922272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] The token cache is being invalidate because project b0c00e8358be425892d62a6e12be20cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:48:58.922555 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.923162 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-7d69cc37-8c1a-4edd-90ab-e5d2616ce5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:48:58.924662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1375/2736] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:48:58 2022] DELETE /identity/v3/projects/b0c00e8358be425892d62a6e12be20cb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:48:58.930242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:58.930753 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:58.930850 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:58.950997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:58.952121 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:59.012750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1362/2737] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:59.020960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.021268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.021544 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.043114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:59.044348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:59.052734 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1376/2738] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:59.057584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.067492 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b3c00-0128-4e5f-b9a4-71e86e2d4c49 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.072550 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.074694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.074694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.074694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.074694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:48:59.075087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.076291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.092291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1363/2739] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:48:59.093634 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.103952 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aacf5a-f863-4bc6-8d23-0e9a44e08c9e None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.108763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.109421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.109694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.109965 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.110621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.112485 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.204293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1377/2740] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:48:59 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:48:59.208154 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.219073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02753302-d02e-4e50-9ccb-812992c13401 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.282823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.283449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.283732 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.284013 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.284671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.286631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.386362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1364/2741] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:48:59 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:48:59.392283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.403504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9fce3-4bcf-47a7-9f63-e6c4b0ec36b2 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.408254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.408867 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.409193 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.409464 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.410157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.411762 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.423509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1378/2742] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:59.427704 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.438591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e94fb5-4aca-4401-8586-a7568ae47223 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.443840 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.444134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.444679 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.445030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.445730 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.447308 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aec464-bb49-4561-8869-2c7b4f0c3d88 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.459445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1365/2743] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:48:59.464538 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.474832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413424-dbff-48ba-945b-673726fe2912 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.479943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.480596 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.480869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.481178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] PATH_INFO: `/v3/projects/4f70a375624d4d3ebc07c04e7d91a0f4/users/2c0e4500efe344bab099dbc8a635d8c3/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.496212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f70a375624d4d3ebc07c04e7d91a0f4, user_id=2c0e4500efe344bab099dbc8a635d8c3, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.498971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.574893 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1379/2744] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:48:59 2022] PUT /identity/v3/projects/4f70a375624d4d3ebc07c04e7d91a0f4/users/2c0e4500efe344bab099dbc8a635d8c3/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:48:59.585448 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.586378 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.586668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.613963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] MFA Rules not processed for user `2c0e4500efe344bab099dbc8a635d8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:48:59.615214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:59.628403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:48:59.682313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1366/2745] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:48:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:48:59.693643 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:48:59.695979 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:48:59.752964 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92239c-f892-418d-a67f-bb7fa9f28749 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:48:59.757202 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:48:59.757878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:48:59.758161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:48:59.758437 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:48:59.766769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:48:59.768188 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24365-374f-4c60-8f9c-fd2d6bbe32fc service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:48:59.777545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1380/2746] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:48:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:09.731021 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=20 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:09.773518 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:16.580302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:16.581744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:16.646215 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63340f-7b7e-471d-8fb8-fc3f9e915aa8 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:16.650740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:16.652104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:16.652385 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:16.652665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] PATH_INFO: `/v3/users/5ff8ff414d374a3dbc7c632d56e34d5e` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:16.653861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff8ff414d374a3dbc7c632d56e34d5e)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:16.655749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:16.827141 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff8ff414d374a3dbc7c632d56e34d5e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:16.838999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff8ff414d374a3dbc7c632d56e34d5e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:16.887641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff8ff414d374a3dbc7c632d56e34d5e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:16.894910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff8ff414d374a3dbc7c632d56e34d5e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:16.895546 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff8ff414d374a3dbc7c632d56e34d5e'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:16.901742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1367/2747] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:16 2022] DELETE /identity/v3/users/5ff8ff414d374a3dbc7c632d56e34d5e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:16.906208 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:16.906698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:16.906814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:16.926807 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:16.928216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:16.989302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1381/2748] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:16.999014 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:17.000463 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:17.059159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ad41-f855-4adc-8ba7-043486ce66a2 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:17.062957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:17.063602 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.063909 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.064185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.073750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:17.075326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ade31-a7d3-41fb-afe4-3871241c1e45 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:17.091440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1368/2749] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:17.091551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:17.091551 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:17.103139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffd019-0eba-4b2b-af77-00114feee5c2 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:17.108677 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:17.109390 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.109686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.109946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] PATH_INFO: `/v3/users/2c0e4500efe344bab099dbc8a635d8c3` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.111105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0e4500efe344bab099dbc8a635d8c3)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:17.113010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:17.325375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0e4500efe344bab099dbc8a635d8c3'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.338238 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0e4500efe344bab099dbc8a635d8c3'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.346862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0e4500efe344bab099dbc8a635d8c3'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.347511 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0e4500efe344bab099dbc8a635d8c3'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.428314 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0e4500efe344bab099dbc8a635d8c3'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.434832 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1382/2750] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:17 2022] DELETE /identity/v3/users/2c0e4500efe344bab099dbc8a635d8c3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:17.439420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.439728 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.440006 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.459082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:17.460263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:17.477064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:17.516479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:17.517918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:17.521916 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1369/2751] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:17.532247 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:17.534194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:17.547987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d8b3-b2e7-4641-a70e-3e8a5a06b01b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:17.552726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:17.553531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.553875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.554114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] PATH_INFO: `/v3/projects/bac941ee247a493daab54f2aee871ebb` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.555283 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac941ee247a493daab54f2aee871ebb)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:17.557287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:17.597517 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9975a-67e2-4c9e-a41b-6e649cafa407 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:17.601235 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:17.601855 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.602134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.602432 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.612027 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:17.613522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:17.623662 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1370/2752] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:49:17.662221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac941ee247a493daab54f2aee871ebb', 'request_id': 'req-10613b80-bedf-4d4a-830b-dda02920fede'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.669429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac941ee247a493daab54f2aee871ebb', 'request_id': 'req-10613b80-bedf-4d4a-830b-dda02920fede'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.725522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac941ee247a493daab54f2aee871ebb', 'request_id': 'req-10613b80-bedf-4d4a-830b-dda02920fede'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.731460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:17.737550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] The token cache is being invalidate because project bac941ee247a493daab54f2aee871ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:17.737841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.738490 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:17.740029 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1383/2753] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:17 2022] DELETE /identity/v3/projects/bac941ee247a493daab54f2aee871ebb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:17.744207 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:17.754031 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceb410-c8f8-457e-8ca2-3e657e6a3878 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:17.812124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:17.812771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:17.813082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:17.813363 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] PATH_INFO: `/v3/users/f2535dd1a0ca48d98ad8680fccac5cb7` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:17.814557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2535dd1a0ca48d98ad8680fccac5cb7)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:17.816520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:18.025699 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2535dd1a0ca48d98ad8680fccac5cb7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.039740 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2535dd1a0ca48d98ad8680fccac5cb7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.042981 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:18.050570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:18.052063 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:18.065973 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613b80-bedf-4d4a-830b-dda02920fede admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:18.070847 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:18.074097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:18.074097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:18.074097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] PATH_INFO: `/v3/projects/4f70a375624d4d3ebc07c04e7d91a0f4` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:18.074097 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f70a375624d4d3ebc07c04e7d91a0f4)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:18.075931 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:18.145069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2535dd1a0ca48d98ad8680fccac5cb7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.152848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2535dd1a0ca48d98ad8680fccac5cb7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.154185 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2535dd1a0ca48d98ad8680fccac5cb7'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.159923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1371/2754] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:17 2022] DELETE /identity/v3/users/f2535dd1a0ca48d98ad8680fccac5cb7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:18.220233 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f70a375624d4d3ebc07c04e7d91a0f4', 'request_id': 'req-ca833541-86fb-46df-bda9-ebe59619caf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.226773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f70a375624d4d3ebc07c04e7d91a0f4', 'request_id': 'req-ca833541-86fb-46df-bda9-ebe59619caf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.351439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f70a375624d4d3ebc07c04e7d91a0f4', 'request_id': 'req-ca833541-86fb-46df-bda9-ebe59619caf7'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.364105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] The token cache is being invalidate because project 4f70a375624d4d3ebc07c04e7d91a0f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:18.364391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.365083 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ca833541-86fb-46df-bda9-ebe59619caf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.366766 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1384/2755] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:18 2022] DELETE /identity/v3/projects/4f70a375624d4d3ebc07c04e7d91a0f4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:18.637481 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:18.638960 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:18.711473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b738ae5-d890-481c-b873-18b8a2100733 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:18.767377 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:18.768036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:18.768313 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:18.768583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] PATH_INFO: `/v3/projects/95f18769f8f54cb3bbf173de668adcdd` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:18.769748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f18769f8f54cb3bbf173de668adcdd)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:18.771638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:18.890938 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f18769f8f54cb3bbf173de668adcdd', 'request_id': 'req-59dacde5-8c29-480a-b3b7-d5464163df63'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.965458 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f18769f8f54cb3bbf173de668adcdd', 'request_id': 'req-59dacde5-8c29-480a-b3b7-d5464163df63'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.971904 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f18769f8f54cb3bbf173de668adcdd', 'request_id': 'req-59dacde5-8c29-480a-b3b7-d5464163df63'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.983103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] The token cache is being invalidate because project 95f18769f8f54cb3bbf173de668adcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:18.983391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.984058 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-59dacde5-8c29-480a-b3b7-d5464163df63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:18.985630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1372/2756] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:18 2022] DELETE /identity/v3/projects/95f18769f8f54cb3bbf173de668adcdd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:18.990542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:18.993297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:18.993297 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.011179 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:19.012373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:19.071465 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1385/2757] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:19.079130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.079447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.079727 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.099362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:19.100561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:19.109347 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1373/2758] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:19.113682 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.123825 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd2cc7-cf31-4eb3-8e81-f7a8779241eb None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.128343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.128951 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.129269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.129546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.130240 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:49:19.130545 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.132123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.143424 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1386/2759] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:49:19.147609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.157194 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701c35e-8411-43a5-883e-7c56cee8a7b7 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.161823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.162442 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.162720 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.162993 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.163654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.165580 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.181451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:19.252391 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1374/2760] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:19.256519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.266802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993a3ef-9a91-4651-b426-1ddb1838d348 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.324070 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.324673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.324948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.325247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.325909 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.327791 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.338426 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:19.437346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1387/2761] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:19.441403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.452409 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed7f5-88e4-4f96-887c-b1a6eaf720b5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.457161 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.457771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.458053 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.458330 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.459030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.460624 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.470749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1375/2762] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:19.474740 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.484637 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa347805-f96b-4828-9b18-315cd4d3db25 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.489431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.490033 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.490311 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.490582 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.491270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.492835 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50abf57-a729-49a8-9d3f-d8fe9fb61c43 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.505201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1388/2763] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:19.508983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.519930 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84e57b-520c-4068-bf37-05ea031a5f05 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.524601 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.525298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.525586 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.525901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] PATH_INFO: `/v3/projects/af031eb97c3e4422a374d5adf140e255/users/fc6650e9a4be41f2a3e1a5d216a71740/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.540303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af031eb97c3e4422a374d5adf140e255, user_id=fc6650e9a4be41f2a3e1a5d216a71740, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.543061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.613167 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1376/2764] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:19 2022] PUT /identity/v3/projects/af031eb97c3e4422a374d5adf140e255/users/fc6650e9a4be41f2a3e1a5d216a71740/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:19.624107 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.624412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.624670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.652723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] MFA Rules not processed for user `fc6650e9a4be41f2a3e1a5d216a71740`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:19.656852 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:19.716959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1389/2765] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:19.721612 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.782671 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cb10c-2278-4345-811f-03b71fb84532 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.787254 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.787872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.788134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.788405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.789116 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.791057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:19.912017 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1377/2766] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:19.916140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:19.927610 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294d9b1-c531-42fc-aea8-addb7ffbe2e0 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:19.988316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:19.989068 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:19.989430 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:19.989787 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:19.990507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:19.992527 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.078739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1390/2767] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:19 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:49:20.082851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:20.093516 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ef288-8ee1-4116-845a-3320771a9839 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:20.098206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:20.098813 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.099093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.099368 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.100075 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:20.101705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.112985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1378/2768] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:20.117267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:20.128066 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e815d45-1042-44e5-94f4-bc4dbbda2ebf admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:20.132667 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:20.133353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.133628 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.138732 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] PATH_INFO: `/v3/projects/d64d837c611843ee9f79cf333aa33dc4/users/ca1a790d4fab40389558a52d013936c9/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.150508 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64d837c611843ee9f79cf333aa33dc4, user_id=ca1a790d4fab40389558a52d013936c9, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:20.153557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.274694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1391/2769] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:20 2022] PUT /identity/v3/projects/d64d837c611843ee9f79cf333aa33dc4/users/ca1a790d4fab40389558a52d013936c9/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:20.279382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:20.289370 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464eecd-e4c9-4427-ae03-b4a924cf682f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:20.346561 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:20.347106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.347388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.347664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.348352 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:20.349976 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c031db3-a883-4b22-b6f9-0af7423bc67d admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.359675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1379/2770] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:20.363968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:20.374775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdd35-f743-4fd6-9ae8-12c6089ddac5 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:20.379301 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:20.379945 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.380225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.380495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] PATH_INFO: `/v3/projects/d64d837c611843ee9f79cf333aa33dc4/users/ca1a790d4fab40389558a52d013936c9/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.382435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64d837c611843ee9f79cf333aa33dc4, user_id=ca1a790d4fab40389558a52d013936c9, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:20.385086 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.399827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1392/2771] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:20 2022] PUT /identity/v3/projects/d64d837c611843ee9f79cf333aa33dc4/users/ca1a790d4fab40389558a52d013936c9/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:20.467773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.468424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.468424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.496931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] MFA Rules not processed for user `ca1a790d4fab40389558a52d013936c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:20.499678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:20.515807 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:20.569095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1380/2772] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:20.581022 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:20.582812 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:20.613936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:20.640410 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162b02-3b7c-4e00-96d9-fa8f15e63284 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:20.645504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:20.645504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:20.645504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:20.645504 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:20.654307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:20.655738 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d60de5-a868-48e1-8d6f-da9cc562559a service cinder] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:20.665267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1393/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:21.573825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:21.636242 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5067-6662-41f3-a63b-9a75cf7b89c4 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:21.640582 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:21.641240 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:21.641519 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:21.641792 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] PATH_INFO: `/v3/users/fc6650e9a4be41f2a3e1a5d216a71740` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:21.642931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6650e9a4be41f2a3e1a5d216a71740)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:21.644832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:21.827174 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6650e9a4be41f2a3e1a5d216a71740'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:21.840834 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6650e9a4be41f2a3e1a5d216a71740'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:21.910559 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6650e9a4be41f2a3e1a5d216a71740'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:21.918654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6650e9a4be41f2a3e1a5d216a71740'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:21.919335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6650e9a4be41f2a3e1a5d216a71740'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:21.925729 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1381/2774] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:21 2022] DELETE /identity/v3/users/fc6650e9a4be41f2a3e1a5d216a71740 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:21.930291 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:21.930767 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:21.930862 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:21.951248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:21.952473 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:22.011190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1394/2775] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:22.020756 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:22.022200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:22.060173 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:22.081988 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf178d-e55b-4899-ab11-06ffd47d6d29 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:22.086214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:22.086362 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:22.086589 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:22.086865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:22.095820 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:22.097286 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:22.107104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1382/2776] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:49:22.530131 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:22.531652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:22.544431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053898c-ea53-4bc6-89a0-175075980b1b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:22.550016 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:22.550743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:22.551024 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:22.551300 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] PATH_INFO: `/v3/projects/af031eb97c3e4422a374d5adf140e255` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:22.552487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=af031eb97c3e4422a374d5adf140e255)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:22.554510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:22.672105 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af031eb97c3e4422a374d5adf140e255', 'request_id': 'req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:22.678317 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af031eb97c3e4422a374d5adf140e255', 'request_id': 'req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:22.773461 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af031eb97c3e4422a374d5adf140e255', 'request_id': 'req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:22.784611 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] The token cache is being invalidate because project af031eb97c3e4422a374d5adf140e255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:22.784902 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:22.785542 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:22.787050 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1395/2777] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:22 2022] DELETE /identity/v3/projects/af031eb97c3e4422a374d5adf140e255 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:22.791029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:22.802057 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05dc7-1000-4aa9-8a8c-70fc5de4e0f5 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:22.858607 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:22.859245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:22.859526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:22.859803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] PATH_INFO: `/v3/users/ca1a790d4fab40389558a52d013936c9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:22.860965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1a790d4fab40389558a52d013936c9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:22.863066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:23.070571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1a790d4fab40389558a52d013936c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.082742 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1a790d4fab40389558a52d013936c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.179124 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1a790d4fab40389558a52d013936c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.187257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1a790d4fab40389558a52d013936c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.187890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a70029ea-492b-46a4-9fae-d9d1979c3956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1a790d4fab40389558a52d013936c9'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.193748 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1383/2778] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:22 2022] DELETE /identity/v3/users/ca1a790d4fab40389558a52d013936c9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:23.610092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:23.611618 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:23.624870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef465-d5ed-4a6d-8d11-8d4c1c7776ab admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:23.683923 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:23.692550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:23.693213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:23.697181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:23.697181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] PATH_INFO: `/v3/projects/d64d837c611843ee9f79cf333aa33dc4` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:23.697181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64d837c611843ee9f79cf333aa33dc4)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:23.697181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:23.836358 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64d837c611843ee9f79cf333aa33dc4', 'request_id': 'req-fac50699-dcac-4741-a86f-335716f94d70'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.842428 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64d837c611843ee9f79cf333aa33dc4', 'request_id': 'req-fac50699-dcac-4741-a86f-335716f94d70'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.949041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64d837c611843ee9f79cf333aa33dc4', 'request_id': 'req-fac50699-dcac-4741-a86f-335716f94d70'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.961335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] The token cache is being invalidate because project d64d837c611843ee9f79cf333aa33dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:23.961621 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.962453 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-fac50699-dcac-4741-a86f-335716f94d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:23.964697 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1396/2779] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:23 2022] DELETE /identity/v3/projects/d64d837c611843ee9f79cf333aa33dc4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:23.970068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:23.970483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:23.971104 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:23.992145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:23.993390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:24.040430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:24.052509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1384/2780] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:24.061100 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.061598 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.061901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.082652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:24.083838 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:24.092008 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1397/2781] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:24.096284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.107087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57608ce6-9aff-48d1-989e-292b125e0abc None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.111848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.112460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.112741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.113061 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.113749 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:49:24.114085 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.115637 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.127615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1385/2782] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:49:24.131816 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.141602 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30807-978f-439a-9384-4deae55ee3a1 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.146224 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.146857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.147135 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.147406 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.148069 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.149984 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.253641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1398/2783] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:24.257890 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.268773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc188b-a2f2-4b38-bffb-d904e3f3858a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.326447 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.327059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.327342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.327620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.328287 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.330228 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.437134 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1386/2784] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:24.441192 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.451928 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a784489-96e8-4fbc-8da4-264f62d288f3 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.456460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.457092 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.457373 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.457687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.458375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.459967 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.470754 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1399/2785] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:24.474590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.485270 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a11c9-4dc8-496a-863e-09270dd4754e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.489896 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.490499 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.490790 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.491069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.491771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.493400 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108a7b-da4a-460b-831f-fb67e2c615e4 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.504221 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1387/2786] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:24.508123 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.517672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f535-895b-4fcc-bc5f-5f24a21551b7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.522221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.522857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.523132 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.523404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] PATH_INFO: `/v3/projects/132ecd9e44704d78801095684e5cfcd8/users/20df5b29f93a4a44a7ebdf20bae01ff2/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.538048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132ecd9e44704d78801095684e5cfcd8, user_id=20df5b29f93a4a44a7ebdf20bae01ff2, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.540649 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.664661 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1400/2787] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:24 2022] PUT /identity/v3/projects/132ecd9e44704d78801095684e5cfcd8/users/20df5b29f93a4a44a7ebdf20bae01ff2/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:24.677698 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.679547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.679547 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.705878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] MFA Rules not processed for user `20df5b29f93a4a44a7ebdf20bae01ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:24.707087 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:24.772032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1388/2788] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:24.777209 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.836870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5305bb-8d6e-4159-aceb-b96ba680fe95 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.842137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:24.842733 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:24.843010 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:24.843280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:24.843939 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:24.845850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:24.928495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1401/2789] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:49:24.932575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:24.943129 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381a5e4-e12c-452f-9ee7-6cb520ec1fb2 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:24.989778 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:25.000261 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.000477 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.000766 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.001082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.001761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.003680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.082096 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1389/2790] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:24 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:25.086017 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:25.092881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:25.096646 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a90929-ec42-4bcc-8ec7-c9824ffff60a admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:25.101344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.101948 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.102228 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.102498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.103191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.105546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.116167 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1402/2791] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:25.121086 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:25.131103 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eea1e-1c71-4247-b40c-0258edc3cc92 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:25.135670 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.136521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.136801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.137102 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] PATH_INFO: `/v3/projects/429647ab39484468be70ef518d1ae4b9/users/bec242e7f267424aa5883c44ea870ae5/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.152744 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=429647ab39484468be70ef518d1ae4b9, user_id=bec242e7f267424aa5883c44ea870ae5, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.155452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.248077 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1390/2792] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:25 2022] PUT /identity/v3/projects/429647ab39484468be70ef518d1ae4b9/users/bec242e7f267424aa5883c44ea870ae5/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:25.252065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:25.262534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4372c1-b93d-4ded-96ac-a5e8cace4654 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:25.319259 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.319875 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.320156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.320434 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.321157 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.322799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ffa77-1470-4516-b34c-b1c432847eaf admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.332470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1403/2793] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:25.336552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:25.347750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a23ea-7be6-4376-8269-24aab0edf29b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:25.352342 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.353159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.353358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.353631 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] PATH_INFO: `/v3/projects/429647ab39484468be70ef518d1ae4b9/users/bec242e7f267424aa5883c44ea870ae5/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.356462 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=429647ab39484468be70ef518d1ae4b9, user_id=bec242e7f267424aa5883c44ea870ae5, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.359081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.369108 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1391/2794] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:25 2022] PUT /identity/v3/projects/429647ab39484468be70ef518d1ae4b9/users/bec242e7f267424aa5883c44ea870ae5/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:25.379195 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.379492 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.379777 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.407456 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] MFA Rules not processed for user `bec242e7f267424aa5883c44ea870ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:25.408696 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:25.476129 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1404/2795] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:25.487427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:25.489163 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:25.546579 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d552bf-fdc1-419e-acdd-f09d44f6af8a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:25.550196 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:25.550854 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:25.551135 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:25.551412 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:25.615868 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:25.617344 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb126f5-f1c7-4487-be02-7401c9a0849c service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:25.626233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1392/2796] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:30.812936 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:30.877173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc1b4-6f03-4017-92ff-cc5789355245 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:30.881723 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:30.882360 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:30.882660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:30.882932 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] PATH_INFO: `/v3/users/20df5b29f93a4a44a7ebdf20bae01ff2` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:30.884065 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20df5b29f93a4a44a7ebdf20bae01ff2)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:30.886606 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:30.902199 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:31.098174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20df5b29f93a4a44a7ebdf20bae01ff2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:31.110670 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20df5b29f93a4a44a7ebdf20bae01ff2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:31.117351 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20df5b29f93a4a44a7ebdf20bae01ff2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:31.118085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20df5b29f93a4a44a7ebdf20bae01ff2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:31.216874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20df5b29f93a4a44a7ebdf20bae01ff2'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:31.223253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1405/2797] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:30 2022] DELETE /identity/v3/users/20df5b29f93a4a44a7ebdf20bae01ff2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:31.227828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:31.228303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:31.228434 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:31.247652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:31.248848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:31.298348 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:31.315700 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1393/2798] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:49:31.325570 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:31.327034 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:31.384803 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff73bb-3aa7-455a-8fe5-2affa3c89c83 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:31.389268 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:31.389884 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:31.390163 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:31.390435 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:31.399421 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:31.400841 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:31.410376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1406/2799] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:49:31.860920 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:31.862520 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:31.875408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb42498-3f10-4136-bfe7-8530108a08db None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:31.880030 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:31.880682 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:31.881137 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:31.881433 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] PATH_INFO: `/v3/projects/132ecd9e44704d78801095684e5cfcd8` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:31.882683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=132ecd9e44704d78801095684e5cfcd8)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:31.884574 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:32.012260 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132ecd9e44704d78801095684e5cfcd8', 'request_id': 'req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.070608 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132ecd9e44704d78801095684e5cfcd8', 'request_id': 'req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.077473 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132ecd9e44704d78801095684e5cfcd8', 'request_id': 'req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.088336 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] The token cache is being invalidate because project 132ecd9e44704d78801095684e5cfcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:32.089303 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.089925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.091510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1394/2800] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:31 2022] DELETE /identity/v3/projects/132ecd9e44704d78801095684e5cfcd8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:32.095687 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:32.106190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a180b-116b-4e80-9e5a-cb5951cc73ee service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:32.163225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:32.163869 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:32.164146 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:32.164419 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] PATH_INFO: `/v3/users/bec242e7f267424aa5883c44ea870ae5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:32.165588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec242e7f267424aa5883c44ea870ae5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:32.168479 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:32.341669 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:32.347460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec242e7f267424aa5883c44ea870ae5'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.359550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec242e7f267424aa5883c44ea870ae5'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.366969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec242e7f267424aa5883c44ea870ae5'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.367675 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec242e7f267424aa5883c44ea870ae5'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.436533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e77fca3b-233f-4809-b040-fd12747c4470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec242e7f267424aa5883c44ea870ae5'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:32.443389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1407/2801] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:32 2022] DELETE /identity/v3/users/bec242e7f267424aa5883c44ea870ae5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:32.837812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:32.839309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:32.853588 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2903-7792-42d3-8d6c-7dd0ea0a7a55 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:32.910522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:32.911172 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:32.911451 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:32.911725 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] PATH_INFO: `/v3/projects/429647ab39484468be70ef518d1ae4b9` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:32.912875 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429647ab39484468be70ef518d1ae4b9)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:32.914803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.041082 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429647ab39484468be70ef518d1ae4b9', 'request_id': 'req-a778dd0a-267b-49ac-80a6-2dc3c217d179'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:33.142687 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429647ab39484468be70ef518d1ae4b9', 'request_id': 'req-a778dd0a-267b-49ac-80a6-2dc3c217d179'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:33.149503 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429647ab39484468be70ef518d1ae4b9', 'request_id': 'req-a778dd0a-267b-49ac-80a6-2dc3c217d179'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:33.163300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] The token cache is being invalidate because project 429647ab39484468be70ef518d1ae4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:33.163587 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:33.164341 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a778dd0a-267b-49ac-80a6-2dc3c217d179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:33.165768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1395/2802] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:32 2022] DELETE /identity/v3/projects/429647ab39484468be70ef518d1ae4b9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:33.170926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.171386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.171595 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.191155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:33.192347 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:33.256178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1408/2803] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:33.264048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.264345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.265253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.285466 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:33.286667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:33.294978 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1396/2804] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:33.299418 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.309413 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080e4a2-b3c0-4a3f-9a9c-19baac8e7d9b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.314981 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.315599 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.315882 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.316156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.316889 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:49:33.317220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.318830 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.331272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1409/2805] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:49:33.335659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.341246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:33.345673 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e069-4da8-45fe-b8f9-2d2904ce56ee None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.350311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.353650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.353650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.353650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.353650 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.354204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.448431 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1397/2806] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:33 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:33.453273 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.463509 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b33f-f678-4cbf-8759-90e01a96e524 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.518478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.519094 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.519375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.519651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.520308 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.522247 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.596111 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1410/2807] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:33 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:33.600557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.611223 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6cc7d-68e5-4eea-b6f3-af86ca2910b9 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.615863 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.616472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.616752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.617059 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.617752 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.619329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.629957 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1398/2808] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:49:33.634139 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.644698 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba2da3-7459-403f-8f22-760072cc8e2e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.649500 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.652974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.652974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.652974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.652974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.652974 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbff50-f31e-4f11-a73b-115d814f5874 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.664114 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1411/2809] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:33.668445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.678551 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f750e1c-18ac-4e16-b2e1-fb06ef34c59b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.683575 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.684244 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.684527 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.684806 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] PATH_INFO: `/v3/projects/e12fd659ab644563affd840b679c1fc6/users/bcfd9bfa662244bf9579e75d4d27c1a2/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.700388 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12fd659ab644563affd840b679c1fc6, user_id=bcfd9bfa662244bf9579e75d4d27c1a2, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.703937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.784358 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1399/2810] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:33 2022] PUT /identity/v3/projects/e12fd659ab644563affd840b679c1fc6/users/bcfd9bfa662244bf9579e75d4d27c1a2/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:33.796374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.797190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.797474 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.825564 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] MFA Rules not processed for user `bcfd9bfa662244bf9579e75d4d27c1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:33.826769 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:33.836180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:33.893265 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1412/2811] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:33.904025 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:33.906003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:33.965026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e88f-211e-4744-a268-81f0ca512a68 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:33.969423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:33.970044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:33.970329 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:33.970605 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:33.979003 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:33.980430 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee3ca-aa02-4aca-ab38-708984c0467b service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:33.989188 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1400/2812] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:34.286533 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:34.287968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:34.353523 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd760e-3686-47eb-b167-9ad2c34f7070 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:34.358060 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:34.358702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:34.358979 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:34.359248 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] PATH_INFO: `/v3/users/36bf876a40df413b90f79fdef7c98726` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:34.360525 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bf876a40df413b90f79fdef7c98726)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:34.362442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:34.539272 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bf876a40df413b90f79fdef7c98726'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:34.551726 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bf876a40df413b90f79fdef7c98726'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:34.562651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bf876a40df413b90f79fdef7c98726'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:34.562651 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bf876a40df413b90f79fdef7c98726'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:34.628036 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bf876a40df413b90f79fdef7c98726'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:34.634588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1413/2813] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:34 2022] DELETE /identity/v3/users/36bf876a40df413b90f79fdef7c98726 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:34.639008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:34.639293 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:34.640206 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:34.660592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:34.661884 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:34.725302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1401/2814] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:49:34.735647 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:34.737293 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:34.774404 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:34.798751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1164-19ac-4f00-a055-2b37d7b2bfc6 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:34.802353 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:34.802959 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:34.803236 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:34.803507 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:34.812845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:34.814324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585bb31-fe9e-4981-9472-e4c5a8e321d3 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:34.823785 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1414/2815] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:35.249768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:35.251943 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:35.264638 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1f7ec-884d-4c3c-a9c8-5ce60c71e175 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:35.269452 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:35.270073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.270356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.270630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] PATH_INFO: `/v3/projects/99b4f140f08e4120a1503c7ecc3e735a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.271816 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b4f140f08e4120a1503c7ecc3e735a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:35.273927 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:35.377558 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:35.380132 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b4f140f08e4120a1503c7ecc3e735a', 'request_id': 'req-ab327eb7-5d5c-444d-9823-8f79eae49da3'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:35.487273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b4f140f08e4120a1503c7ecc3e735a', 'request_id': 'req-ab327eb7-5d5c-444d-9823-8f79eae49da3'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:35.494309 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b4f140f08e4120a1503c7ecc3e735a', 'request_id': 'req-ab327eb7-5d5c-444d-9823-8f79eae49da3'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:35.505714 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] The token cache is being invalidate because project 99b4f140f08e4120a1503c7ecc3e735a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:35.506005 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:35.506651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-ab327eb7-5d5c-444d-9823-8f79eae49da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:35.508177 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1402/2816] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:35 2022] DELETE /identity/v3/projects/99b4f140f08e4120a1503c7ecc3e735a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:35.513174 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.514721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.514721 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.533220 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:35.534432 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:35.594946 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1415/2817] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:35.662690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.663067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.663288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.684545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:35.686951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:35.694630 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1403/2818] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:35.698452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:35.708310 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078eb12-ce69-40de-ad09-0606097104ae None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:35.713052 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:35.715436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.715436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.715436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.715436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:49:35.715436 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:35.716742 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:35.732605 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1416/2819] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:49:35.734901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:35.748325 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f04e-019b-40dd-8407-c75841c6c8a3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:35.749114 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:35.750001 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.750284 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.750564 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.751218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:35.753149 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:35.844106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1404/2820] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:35 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:35.848438 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:35.859177 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cd240-67e1-42a7-937e-6503205708c4 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:35.916705 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:35.917287 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:35.917569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:35.917845 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:35.918497 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:35.920702 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:36.042296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1417/2821] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:35 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:36.047000 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:36.057507 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862642d-0a47-449f-a525-36326ab97fb5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:36.062627 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:36.063291 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:36.063656 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:36.063939 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:36.064741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:36.066399 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:36.077675 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1405/2822] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:36.081978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:36.094331 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6748e-eb2d-4ef8-85fd-e909699a6c23 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:36.099941 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:36.100552 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:36.100911 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:36.101222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:36.101906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:36.103481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33bc4-64a1-41d4-a829-41e029efdeec admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:36.114306 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1418/2823] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:49:36.118556 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:36.128647 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b0f3-86d1-4f42-a0c1-d98a83b7dc3a admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:36.133333 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:36.133989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:36.140331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:36.140331 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] PATH_INFO: `/v3/projects/5d82f0f1c0a445d1b6c3b7b363c096be/users/ca3a3b1e03d5407ab19395fb4c0f2b91/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:36.149937 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d82f0f1c0a445d1b6c3b7b363c096be, user_id=ca3a3b1e03d5407ab19395fb4c0f2b91, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:36.152683 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:36.239872 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1406/2824] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:36 2022] PUT /identity/v3/projects/5d82f0f1c0a445d1b6c3b7b363c096be/users/ca3a3b1e03d5407ab19395fb4c0f2b91/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:36.249956 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99607d-998e-499c-80c6-557c76955d7b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:36.250304 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99607d-998e-499c-80c6-557c76955d7b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:36.250583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99607d-998e-499c-80c6-557c76955d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:36.282130 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-8c99607d-998e-499c-80c6-557c76955d7b None None] MFA Rules not processed for user `ca3a3b1e03d5407ab19395fb4c0f2b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:36.283343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-8c99607d-998e-499c-80c6-557c76955d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:36.314804 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:36.352567 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1419/2825] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:36.363997 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:36.365736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:36.426455 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f8b22-9716-48ab-b291-67d3b3ec821e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:36.430243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:36.430888 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:36.431170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:36.431445 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:36.440536 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:36.442055 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:36.450869 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1407/2826] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:39.261154 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:39.261729 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:39.261916 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:39.282737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] MFA Rules not processed for user `ca3a3b1e03d5407ab19395fb4c0f2b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:39.283937 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:39.291012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1420/2827] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:39.830773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:39.832750 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:39.919876 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d07c-70ae-4d23-9d9b-1bb303112a70 service cinder] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:39.926273 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:39.926895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:39.927183 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:39.927491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:39.929250 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:39.941590 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:39.943170 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:39.953048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1408/2828] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:41.308813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:41.310817 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:41.326863 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5988daa-ad64-47f9-992b-269b0e8fec65 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:41.332601 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:41.333339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:41.333690 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:41.334267 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:41.336225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:41.349553 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:41.351173 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d5fa3-80f0-4502-a398-75fa58d7ca4a service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:41.362035 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1421/2829] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:45.437970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:45.439905 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:45.445552 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:45.508878 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011170e2-efc0-4c6d-9fe8-b3548d96607f service placement] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:45.513669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:45.514356 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:45.514643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:45.514923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] PATH_INFO: `/v3/users/bcfd9bfa662244bf9579e75d4d27c1a2` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:45.516926 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfd9bfa662244bf9579e75d4d27c1a2)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:45.518913 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:45.747073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfd9bfa662244bf9579e75d4d27c1a2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:45.759864 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfd9bfa662244bf9579e75d4d27c1a2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:45.824553 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfd9bfa662244bf9579e75d4d27c1a2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:45.831953 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfd9bfa662244bf9579e75d4d27c1a2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:45.832563 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfd9bfa662244bf9579e75d4d27c1a2'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:45.838541 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1409/2830] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:49:45 2022] DELETE /identity/v3/users/bcfd9bfa662244bf9579e75d4d27c1a2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:49:45.843011 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:45.843460 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:45.843607 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:45.862614 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:45.863808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:45.920799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1422/2831] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:45.931186 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:45.932571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:45.990496 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f286e43-1240-4859-af8d-07d8e682c2af admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:45.994166 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:45.994818 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:45.995101 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:45.995376 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.005494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:46.006961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7564-df66-47fb-aaf9-adebfe3a7645 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:46.016593 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1410/2832] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:46.426815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:46.428295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:46.440577 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf3a7e-34f9-4e41-ba6a-94d3aa40887b None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:46.445280 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:46.445969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:46.446307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:46.446583 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] PATH_INFO: `/v3/projects/e12fd659ab644563affd840b679c1fc6` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.448037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12fd659ab644563affd840b679c1fc6)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:46.449725 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:46.586828 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12fd659ab644563affd840b679c1fc6', 'request_id': 'req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:46.593316 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12fd659ab644563affd840b679c1fc6', 'request_id': 'req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:46.656958 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12fd659ab644563affd840b679c1fc6', 'request_id': 'req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:46.668870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] The token cache is being invalidate because project e12fd659ab644563affd840b679c1fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:49:46.669189 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:46.670113 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-0fe03fee-576e-4de9-aed2-6ac80b94b2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:49:46.671786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1423/2833] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:49:46 2022] DELETE /identity/v3/projects/e12fd659ab644563affd840b679c1fc6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:46.677026 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:46.677530 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:46.677641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.696542 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:46.697794 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:46.717908 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:46.759859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1411/2834] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:46.768779 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:46.769108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:46.769391 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.789487 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:46.790692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:46.798969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1424/2835] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:46.803316 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:46.813418 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e29b8-20a0-4901-9525-f20ec96e1775 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:46.818038 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:46.818651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:46.818936 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:46.819220 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] PATH_INFO: `/v3/domains` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.819910 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80331) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:49:46.820218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:46.821851 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:46.834042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1412/2836] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 29 14:49:46.838302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:46.847813 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d12d7-eaeb-4ac0-82f3-58f70900ccf8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:46.852378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:46.852978 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:46.853289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:46.853563 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:46.854226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:46.856118 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:46.951296 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1425/2837] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:46 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:46.955996 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:46.965915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806fc96-b281-45c7-be50-ac9f442b5558 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:47.023567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:47.024181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.029335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.029335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] PATH_INFO: `/v3/users` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.029335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:47.029335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:47.164758 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1413/2838] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:46 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:47.168799 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:47.177374 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:47.183626 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85251b9-dff1-4353-abf0-8b6d3b5787e7 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:47.187257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:47.187879 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.188275 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.188627 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.189376 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:47.191040 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:47.201377 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1426/2839] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:47.205809 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:47.216119 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838dd54-a8e6-4d01-8c63-8d1a21e5693b admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:47.222332 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:47.222950 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.223414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.223621 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.224320 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:47.225929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b03cd-d1ec-4a5a-846f-b79146af94b9 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:47.237498 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1414/2840] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:47.241784 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:47.252389 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb566cce-5446-483f-825a-75b32f7c413b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:47.257269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:47.258057 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.258375 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.258737 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] PATH_INFO: `/v3/projects/be1df25ae6404f33b88dda96469e9a7e/users/24992429cbbd47f3958e0de1be93916a/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.274270 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1df25ae6404f33b88dda96469e9a7e, user_id=24992429cbbd47f3958e0de1be93916a, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:47.277190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:47.362720 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1427/2841] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:47 2022] PUT /identity/v3/projects/be1df25ae6404f33b88dda96469e9a7e/users/24992429cbbd47f3958e0de1be93916a/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:47.367886 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.368262 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.368615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.387801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:47.389117 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:47.454826 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1415/2842] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:47.461745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:47.463793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:47.522279 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8acb8d-4dd0-4425-9e82-1b0c5e413b97 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:47.526058 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:47.526691 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:47.526975 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:47.527252 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:47.537009 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:47.538444 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986b897-bd76-4d81-bc86-ace51acba05c service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:47.548090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1428/2843] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:50.729150 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:50.731360 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:50.771652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:50.821652 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d82d-85f8-42a6-857b-c7b39e5e03de None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:50.827791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:50.829157 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:50.829619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:50.830068 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:50.832298 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:50.850789 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:50.854036 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:50.877239 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1416/2844] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:49:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:49:53.379815 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:53.380315 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:53.380431 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:53.415241 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] MFA Rules not processed for user `24992429cbbd47f3958e0de1be93916a`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:49:53.416541 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:49:53.486617 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1429/2845] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:49:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:49:53.491304 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:53.502361 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a6394-3fb7-445c-9356-fe2eb73acbe6 service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:53.506836 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:53.507476 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:53.507757 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:53.508032 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:53.508689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:53.510623 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:53.606619 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1417/2846] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:49:53 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:53.607513 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:53.621748 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888f8da-7a8e-400f-8cdf-26d5a6eba313 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:53.681790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:53.682412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:53.682692 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:53.682969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:53.683632 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:53.685662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:53.695324 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:49:53.811178 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1430/2847] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:49:53 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:49:53.815246 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:53.826345 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b17d36-c710-4b63-841c-8c0580cec652 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:53.831040 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:53.831667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:53.831952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:53.832232 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:53.832942 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:53.834648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:53.847741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1418/2848] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:53.850870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:53.862326 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262ca7d-5a64-4592-a2cb-a94e795cb693 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:53.867038 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:53.867743 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:53.868025 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:53.868302 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] PATH_INFO: `/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1/users/03b0466e477c421aac4d70971cf01532/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:53.883344 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f78b0bf8e2d4154aa1bd59e91aa95f1, user_id=03b0466e477c421aac4d70971cf01532, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:53.886028 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:53.961055 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1431/2849] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:53 2022] PUT /identity/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1/users/03b0466e477c421aac4d70971cf01532/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:49:53.965288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:53.975929 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c776e-5aaa-4a09-b3ed-95fc1e346780 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:54.037253 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:54.038002 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:54.038175 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:54.038454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:54.039159 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:54.040771 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27305ef-0fd8-4126-bd93-7e82ed0cb1ee admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:54.053446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1419/2850] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:49:54.057885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:49:54.071773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36bd6-71bf-4aae-b63f-c1ac5a939628 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:49:54.076673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:49:54.077386 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:49:54.077668 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:49:54.077943 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] PATH_INFO: `/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1/users/03b0466e477c421aac4d70971cf01532/roles/744756d2b40c45a2aa4ea4c4d207d3f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:49:54.080246 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f78b0bf8e2d4154aa1bd59e91aa95f1, user_id=03b0466e477c421aac4d70971cf01532, role_id=744756d2b40c45a2aa4ea4c4d207d3f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:49:54.082955 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:49:54.093907 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1432/2851] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:49:54 2022] PUT /identity/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1/users/03b0466e477c421aac4d70971cf01532/roles/744756d2b40c45a2aa4ea4c4d207d3f1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:00.390046 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4d338-ec86-4b6d-a186-b17da38363c4 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:00.391461 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4d338-ec86-4b6d-a186-b17da38363c4 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:00.392069 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4d338-ec86-4b6d-a186-b17da38363c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:00.427459 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-eca4d338-ec86-4b6d-a186-b17da38363c4 None None] MFA Rules not processed for user `03b0466e477c421aac4d70971cf01532`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:00.428825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-eca4d338-ec86-4b6d-a186-b17da38363c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:00.526895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1420/2852] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:00.541090 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:00.544913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:00.625510 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef083e1-0deb-432e-aac7-4b75790e54a8 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:00.631201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:00.631877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:00.632156 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:00.632429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:00.695348 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:00.696971 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec79122-ef27-45d8-971b-41b94eee2ee9 service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:00.706468 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1433/2853] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:03.662531 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128267a3-cc7f-4dc5-9cf0-afa9ba9e62f3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:03.663584 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128267a3-cc7f-4dc5-9cf0-afa9ba9e62f3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:03.663862 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128267a3-cc7f-4dc5-9cf0-afa9ba9e62f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:03.672761 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:03.719357 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:03.722686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:03.722686 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:03.751217 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-128267a3-cc7f-4dc5-9cf0-afa9ba9e62f3 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:03.752521 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-128267a3-cc7f-4dc5-9cf0-afa9ba9e62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:03.758793 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] MFA Rules not processed for user `24992429cbbd47f3958e0de1be93916a`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:03.759999 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:03.775653 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1434/2854] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 29 14:50:03.864917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1421/2855] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:04.061773 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:04.064085 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:04.169728 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:04.231481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcda7e-704f-4f4f-b7ca-e37dd0db874a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:04.242478 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:04.243216 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:04.243993 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:04.244278 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:04.266704 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:04.268739 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:04.279225 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1435/2856] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:50:05.221130 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23afb-78cd-4992-b773-64e49c1aa9ec None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:05.221635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23afb-78cd-4992-b773-64e49c1aa9ec None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:05.221779 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23afb-78cd-4992-b773-64e49c1aa9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:05.245203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b6b23afb-78cd-4992-b773-64e49c1aa9ec None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:05.246501 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b6b23afb-78cd-4992-b773-64e49c1aa9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:05.254597 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1422/2857] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:05.268075 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:05.279635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0aed4-631c-4138-828f-b08527be969b service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:05.286586 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:05.287370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:05.287757 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:05.288137 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:05.297966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:05.299546 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:05.307887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1436/2858] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:50:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:50:05.980848 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c27379-6364-4eb6-aedb-a3151daf13bc None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:05.981111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c27379-6364-4eb6-aedb-a3151daf13bc None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:05.981389 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c27379-6364-4eb6-aedb-a3151daf13bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:06.004924 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-19c27379-6364-4eb6-aedb-a3151daf13bc None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:06.006095 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-19c27379-6364-4eb6-aedb-a3151daf13bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:06.018690 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1423/2859] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Jul 29 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:06.035043 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:06.052341 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cfcb0-8756-4a6d-859c-7f7b2412d3d1 service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:06.058673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:06.058673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:06.059002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:06.059336 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:06.068908 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:06.070631 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:06.080319 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1437/2860] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:50:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 29 14:50:06.715824 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:06.716335 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:06.716632 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:06.737628 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] MFA Rules not processed for user `b4710cdb68364a5696a2e33ad6232650`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:06.739106 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:06.749572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1424/2861] 10.222.0.49 () {58 vars in 967 bytes} [Fri Jul 29 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:06.762837 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:06.773976 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8a1a-4de8-47d7-9091-8cbd4df37f24 service swift] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:06.779155 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3189acc363fe42339aedae0a604c8b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:06.779753 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:06.780067 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:06.780343 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:06.789299 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:06.790881 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4554-1257-4caf-a178-a1864aa609b7 service swift] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:06.800119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1438/2862] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Jul 29 14:50:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 29 14:50:09.667311 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:09.671634 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:09.741379 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba720fc-11db-4d88-b8af-07f9e71517f3 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:09.746073 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:09.746726 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:09.747009 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:09.747288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] PATH_INFO: `/v3/users/ca3a3b1e03d5407ab19395fb4c0f2b91` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:09.748490 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3a3b1e03d5407ab19395fb4c0f2b91)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:09.750460 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:09.933263 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3a3b1e03d5407ab19395fb4c0f2b91'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:09.941011 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:09.945847 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3a3b1e03d5407ab19395fb4c0f2b91'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.027710 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3a3b1e03d5407ab19395fb4c0f2b91'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.036209 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3a3b1e03d5407ab19395fb4c0f2b91'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.036952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3a3b1e03d5407ab19395fb4c0f2b91'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.042989 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1425/2863] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:50:09 2022] DELETE /identity/v3/users/ca3a3b1e03d5407ab19395fb4c0f2b91 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:10.048600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070af97-5980-4553-b008-77724d6026e8 None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.048600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070af97-5980-4553-b008-77724d6026e8 None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.048600 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070af97-5980-4553-b008-77724d6026e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.068382 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-3070af97-5980-4553-b008-77724d6026e8 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:10.069604 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3070af97-5980-4553-b008-77724d6026e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.114648 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:10.116181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.174269 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1439/2864] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:10.180281 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0693e7c-c869-4190-bd7e-84cb7bf372d0 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:10.184175 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070af97-5980-4553-b008-77724d6026e8 None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:10.184423 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:10.185066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.185353 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.185629 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.185741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3070af97-5980-4553-b008-77724d6026e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.187616 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.198714 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070af97-5980-4553-b008-77724d6026e8 None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:10.200961 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:10.202346 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:10.202555 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:10.202969 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.203254 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.203531 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.213442 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:10.213923 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1426/2865] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:10.214877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a516abd-e8bb-4fd6-8bb6-2db9fafed27e service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:10.225440 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1440/2866] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 29 14:50:10.498424 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:10.500008 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.513773 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286356-f6e7-4957-9f8f-386b4bec792d service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:10.518689 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:10.519557 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.519952 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.520326 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] PATH_INFO: `/v3/projects/5d82f0f1c0a445d1b6c3b7b363c096be` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.521941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d82f0f1c0a445d1b6c3b7b363c096be)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:10.523968 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:10.640949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d82f0f1c0a445d1b6c3b7b363c096be', 'request_id': 'req-e3c5345b-cbad-481f-b627-2856ff6136b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.721803 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d82f0f1c0a445d1b6c3b7b363c096be', 'request_id': 'req-e3c5345b-cbad-481f-b627-2856ff6136b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.730020 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d82f0f1c0a445d1b6c3b7b363c096be', 'request_id': 'req-e3c5345b-cbad-481f-b627-2856ff6136b8'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.742044 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] The token cache is being invalidate because project 5d82f0f1c0a445d1b6c3b7b363c096be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:50:10.742233 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.742903 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e3c5345b-cbad-481f-b627-2856ff6136b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:10.744659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1427/2867] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:50:10 2022] DELETE /identity/v3/projects/5d82f0f1c0a445d1b6c3b7b363c096be => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:10.749822 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.750180 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.750524 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.777350 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:10.778557 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.844929 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1441/2868] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:10.853094 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.853390 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.853669 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.874035 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:10.875258 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:10.883279 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1428/2869] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:10.887493 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:10.892335 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:10.898051 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67779d5-83c8-41d7-aa4d-e3d4e2570d0f None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:10.902926 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:10.903565 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.903860 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.904125 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] PATH_INFO: `/v3/domains` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.904850 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80330) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 29 14:50:10.905204 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:10.906842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:10.924338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1442/2870] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Jul 29 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 29 14:50:10.925768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:10.935145 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d736d6-d9ee-4f23-b550-71dfc1b0f118 None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:10.939951 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:10.940571 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:10.940852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:10.941160 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:10.942037 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:10.943745 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.043866 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1429/2871] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Jul 29 14:50:10 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 29 14:50:11.048048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:11.059652 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654b914-6dd3-465d-b628-8cdd8f1b280b admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:11.118289 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:11.118920 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.119201 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.119481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] PATH_INFO: `/v3/users` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.120161 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:11.122119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.222181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1443/2872] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Jul 29 14:50:11 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 29 14:50:11.223545 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:11.234895 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f776-a7df-4abf-894d-4a766aa8a3b3 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:11.240591 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:11.241478 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.241805 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.242179 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.242999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:11.244705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.256965 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1430/2873] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 29 14:50:11.260854 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:11.281713 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400792a-2794-491c-9ebc-01cc7ebb035e admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:11.287899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:11.288913 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.289495 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.289774 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.290846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:11.292750 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3ab5e-b7e9-48ff-963c-5be493f6ff6e admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.306023 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1444/2874] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Jul 29 14:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 29 14:50:11.310355 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:11.322483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33d1c-00eb-43b9-a5f2-759c8bfb3bb5 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:11.328615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:11.328615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.329248 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.330195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] PATH_INFO: `/v3/projects/dc6ddeb0546943f99c7e8610686183f5/users/bc265dfa04054c7ca16bdabed4146dd8/roles/680f93d19a99495a81d0cf39e7ac3ce5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.349051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6ddeb0546943f99c7e8610686183f5, user_id=bc265dfa04054c7ca16bdabed4146dd8, role_id=680f93d19a99495a81d0cf39e7ac3ce5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:11.351931 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.411508 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:11.417763 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1431/2875] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Jul 29 14:50:11 2022] PUT /identity/v3/projects/dc6ddeb0546943f99c7e8610686183f5/users/bc265dfa04054c7ca16bdabed4146dd8/roles/680f93d19a99495a81d0cf39e7ac3ce5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:11.423211 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5269a-11de-4491-a10c-7f005dd669bb None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.423660 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5269a-11de-4491-a10c-7f005dd669bb None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.424184 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5269a-11de-4491-a10c-7f005dd669bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.443820 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-bff5269a-11de-4491-a10c-7f005dd669bb None None] MFA Rules not processed for user `4c8c97aa5a724434b6ecf6c70f044b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:11.445458 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-bff5269a-11de-4491-a10c-7f005dd669bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:11.509221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1445/2876] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:11.520596 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:11.523076 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:11.584165 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bab1-200a-4014-8f6f-e6891431cb56 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:11.589048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:11.593705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:11.593705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:11.593705 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:11.601419 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:11.602941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:11.612615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1432/2877] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:13.171002 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] REQUEST_METHOD: `POST` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:13.171899 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:13.172534 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:13.261339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.auth.core [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] MFA Rules not processed for user `bc265dfa04054c7ca16bdabed4146dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80330) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:13.261339 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:13.403751 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:13.410814 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1446/2878] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:13.430746 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:13.433029 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:13.551243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36e8c-97ca-4499-a056-b35bca19a1ee service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:13.555659 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7c0f85f0b514af0bc2de51aeaf4ffdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:13.556532 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:13.556668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:13.557144 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:13.573195 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:13.574654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea79ce7-96e0-4088-bfda-8827c5ba900f service cinder] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:13.590212 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1433/2879] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:14.172934 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:14.174672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:14.248901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23797089-638c-42d1-9182-7d9a128dc36a None None] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:14.254226 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:14.254987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:14.255342 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:14.255707 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:14.257717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:14.285281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:14.286877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:14.298644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1447/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:16.463919 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:16.464440 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:16.464567 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:16.483302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] MFA Rules not processed for user `bc265dfa04054c7ca16bdabed4146dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:50:16.484828 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:16.492755 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1434/2881] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:50:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:50:16.812271 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:16.813706 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:16.825007 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d84a98-300a-4e0a-a8bd-29c26d4dbc1f service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:16.828588 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:16.829231 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:16.829655 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:16.829938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:16.831836 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:16.843213 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:16.844673 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:16.854074 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1448/2882] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:17.921243 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:17.922823 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:17.943537 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e96aa-f2bc-4e4e-8ab4-9db9b61422cd None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:17.945832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:17.946549 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:17.946891 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:17.947237 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] PATH_INFO: `/v3/users/f9360a76354e4ae99668fe8debe4e7fb` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:17.962327 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9360a76354e4ae99668fe8debe4e7fb)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:17.964283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:18.171812 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9360a76354e4ae99668fe8debe4e7fb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.184703 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9360a76354e4ae99668fe8debe4e7fb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.321427 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9360a76354e4ae99668fe8debe4e7fb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.330654 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9360a76354e4ae99668fe8debe4e7fb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.331387 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9360a76354e4ae99668fe8debe4e7fb'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.332814 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:18.338078 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1435/2883] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:50:17 2022] DELETE /identity/v3/users/f9360a76354e4ae99668fe8debe4e7fb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:18.651470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:18.667824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50340a48-ce07-4c05-9ce3-168ccbed3133 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:18.731711 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:18.732359 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:18.732644 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:18.732918 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] PATH_INFO: `/v3/projects/10f1fba5db2b4402b2fe8d12d099e331` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:18.744445 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f1fba5db2b4402b2fe8d12d099e331)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:18.746400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:18.856219 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f1fba5db2b4402b2fe8d12d099e331', 'request_id': 'req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.863439 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f1fba5db2b4402b2fe8d12d099e331', 'request_id': 'req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.936402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f1fba5db2b4402b2fe8d12d099e331', 'request_id': 'req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.942615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:18.949491 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] The token cache is being invalidate because project 10f1fba5db2b4402b2fe8d12d099e331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:50:18.949786 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.950402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:18.951919 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1449/2884] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:50:18 2022] DELETE /identity/v3/projects/10f1fba5db2b4402b2fe8d12d099e331 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:18.956184 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:18.957664 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:18.971667 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447d74-bb23-4435-8468-de05fbc80b9f admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:19.028985 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:19.029665 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:19.029949 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:19.030251 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] PATH_INFO: `/v3/users/fe5ed1be57264d96834671e0d31747cc` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:19.045526 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5ed1be57264d96834671e0d31747cc)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:19.047515 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:19.244721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5ed1be57264d96834671e0d31747cc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.256741 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5ed1be57264d96834671e0d31747cc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.332603 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5ed1be57264d96834671e0d31747cc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.339944 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5ed1be57264d96834671e0d31747cc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.340592 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5ed1be57264d96834671e0d31747cc'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.347472 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1436/2885] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:50:18 2022] DELETE /identity/v3/users/fe5ed1be57264d96834671e0d31747cc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:50:19.629625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:19.629625 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:19.644871 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0fe9-00b2-44bf-b9a5-43610fae9137 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:19.704222 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:19.704857 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:19.709636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:19.709636 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] PATH_INFO: `/v3/projects/85ea9dc0532d4b02a304af748e008b96` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:19.720400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ea9dc0532d4b02a304af748e008b96)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:19.723672 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:19.820689 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ea9dc0532d4b02a304af748e008b96', 'request_id': 'req-ab28e48c-1397-4dae-8bac-0a2332f2a406'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.829256 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ea9dc0532d4b02a304af748e008b96', 'request_id': 'req-ab28e48c-1397-4dae-8bac-0a2332f2a406'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.875827 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ea9dc0532d4b02a304af748e008b96', 'request_id': 'req-ab28e48c-1397-4dae-8bac-0a2332f2a406'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.889746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] The token cache is being invalidate because project 85ea9dc0532d4b02a304af748e008b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:50:19.890041 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.890695 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:50:19.892469 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1450/2886] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:50:19 2022] DELETE /identity/v3/projects/85ea9dc0532d4b02a304af748e008b96 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 29 14:50:29.343081 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:29.887878 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:33.884533 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:33.886039 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:33.944583 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a31ca6-1b75-4ffa-9381-480c5c7d3abd admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:33.993969 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:33.994585 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:33.994867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:33.995168 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:33.996963 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:34.071051 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:34.072522 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:34.083680 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1437/2887] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:35.266470 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:35.267842 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:35.280187 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e48c-1397-4dae-8bac-0a2332f2a406 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:35.283797 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:35.284412 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:35.284688 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:35.284960 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:35.286710 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:35.300452 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:35.301886 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:35.313498 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1451/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:44.078572 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=74 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:45.308363 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:46.389693 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:46.391115 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:46.402449 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c095a0d-1f61-43b9-9282-ae43d6f7fec6 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:46.406064 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:46.406711 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:46.407042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:46.407288 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:46.408975 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:46.470837 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:46.472255 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a769b5-ff5e-414b-a671-06a7a7864b0c service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:46.481906 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1438/2889] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:48.389234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:50:48.390671 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:48.405868 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81f8e-a143-4c74-9487-d9fe44bd1672 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:50:48.411717 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:50:48.412338 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:50:48.412615 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:50:48.412887 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:50:48.415307 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:50:48.427938 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:50:48.429550 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:50:48.441181 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1452/2890] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:50:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:50:56.477554 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=30 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:50:58.436140 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:51:43.074373 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] REQUEST_METHOD: `POST` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:51:43.074925 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:51:43.075042 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:51:43.094435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.auth.core [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] MFA Rules not processed for user `bc265dfa04054c7ca16bdabed4146dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80331) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 29 14:51:43.095617 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:51:43.149548 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1439/2891] 10.222.0.49 () {58 vars in 958 bytes} [Fri Jul 29 14:51:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 29 14:51:46.476635 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:51:46.486792 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab24e5f-ad89-4dea-b839-dedca265755a service nova] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:51:46.490395 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:51:46.491000 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:51:46.491281 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:51:46.491559 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:51:46.493329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:51:46.503519 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:51:46.504935 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:51:46.512924 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1453/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:51:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:51:53.142917 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=25 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:51:56.509294 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=6 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:52:27.082300 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:52:27.083845 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:52:27.101147 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621168b-3ed2-4285-ae9d-a93e11c221df None None] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:52:27.106865 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a3cbcc38d741178be408abd4156959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:52:27.107510 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:52:27.107791 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:52:27.108066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:52:27.109849 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:52:27.122609 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:52:27.124191 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:52:27.135074 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1440/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:52:37.130414 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=6 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:52:56.530874 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:52:56.532305 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:52:56.599328 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dee68-6de1-4dd1-8ef2-fcb4a6ba5806 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:52:56.603885 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:52:56.604526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:52:56.604805 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:52:56.605119 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] PATH_INFO: `/v3/users/bc265dfa04054c7ca16bdabed4146dd8` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:52:56.606295 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc265dfa04054c7ca16bdabed4146dd8)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:52:56.608288 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:52:56.828790 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc265dfa04054c7ca16bdabed4146dd8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:56.841712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc265dfa04054c7ca16bdabed4146dd8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:56.848569 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc265dfa04054c7ca16bdabed4146dd8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:56.849329 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc265dfa04054c7ca16bdabed4146dd8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:56.902212 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc265dfa04054c7ca16bdabed4146dd8'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:56.908318 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1454/2894] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:52:56 2022] DELETE /identity/v3/users/bc265dfa04054c7ca16bdabed4146dd8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:52:57.141525 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:52:57.153678 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b693ec-b0b3-4b0b-8d91-655446d8a5f4 service nova] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:52:57.213941 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:52:57.214576 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:52:57.214859 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:52:57.215138 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] PATH_INFO: `/v3/projects/dc6ddeb0546943f99c7e8610686183f5` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:52:57.216280 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6ddeb0546943f99c7e8610686183f5)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:52:57.218214 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:52:57.342045 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6ddeb0546943f99c7e8610686183f5', 'request_id': 'req-3db345d9-c4a4-47ed-9c97-6a80edf22a01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:57.393713 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6ddeb0546943f99c7e8610686183f5', 'request_id': 'req-3db345d9-c4a4-47ed-9c97-6a80edf22a01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:57.400454 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6ddeb0546943f99c7e8610686183f5', 'request_id': 'req-3db345d9-c4a4-47ed-9c97-6a80edf22a01'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:57.411153 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] The token cache is being invalidate because project dc6ddeb0546943f99c7e8610686183f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80331) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:52:57.411435 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:57.412089 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:52:57.413643 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1441/2895] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:52:57 2022] DELETE /identity/v3/projects/dc6ddeb0546943f99c7e8610686183f5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:53:06.904658 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:53:07.407599 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:53:48.414115 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:53:48.415914 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:53:48.525944 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7ba78-deb2-409f-a81e-f8f5ffbe0d4d admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:53:48.574870 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:53:48.575483 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:53:48.575763 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:53:48.576037 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:53:48.577824 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:53:48.587234 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:53:48.588641 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:53:48.597245 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1455/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:53:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:53:48.678382 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:53:48.679799 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:53:48.695049 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db345d9-c4a4-47ed-9c97-6a80edf22a01 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:53:48.698560 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:53:48.699218 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:53:48.699494 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:53:48.699768 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:53:48.701485 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:53:48.798487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:53:48.799901 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:53:48.810852 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1442/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:53:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:53:58.593694 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=59 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:53:58.805808 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=48 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:54:55.953966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:54:55.955369 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:54:56.045392 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867dad-1fe5-4661-95d5-8ea28ff6e3a4 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:54:56.051099 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:54:56.051712 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:54:56.051987 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:54:56.052257 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:54:56.054012 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:54:56.066463 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:54:56.071475 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:54:56.078466 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1456/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:55:06.073826 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=41 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:55:50.685801 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:55:50.704598 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce031274-e765-451b-815e-3df732f786c7 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:55:50.708974 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:55:50.709646 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:55:50.709932 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:55:50.710213 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:55:50.711962 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:55:50.769883 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:55:50.771915 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:55:50.780983 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1443/2899] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:55:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:55:52.308968 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:55:52.319136 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c143c4-c868-4134-9bda-407d6f30e26e service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:55:52.322806 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:55:52.323496 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:55:52.323846 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:55:52.324190 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:55:52.326056 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:55:52.353367 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:55:52.354808 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:55:52.376370 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1457/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:55:53.561181 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:55:53.562645 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:55:53.587374 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de96649-39f6-4b33-bc8f-1ceb9d3b0d40 service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:55:53.591200 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:55:53.591825 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:55:53.592111 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:55:53.592384 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:55:53.594491 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:55:53.611736 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:55:53.613202 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:55:53.625668 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1444/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:55:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:55:57.349966 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:55:57.358746 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414508f-5704-4e5e-bda5-671940ba65e4 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:55:57.362402 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06cce26ee600472f84a9d8f7b782bdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:55:57.362989 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:55:57.363266 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:55:57.363536 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:55:57.365400 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:55:57.450230 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:55:57.451662 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:55:57.470866 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1458/2902] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:56:00.808511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:00.808511 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:00.825651 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf61b8-d4ad-414d-b426-7330b6e7590d service neutron] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:00.830838 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:00.836861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:00.836861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:00.836861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:00.836861 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:00.847408 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:00.849066 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:00.859487 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1445/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:56:04.284898 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:04.298027 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548aed0-3a2c-4f46-9b31-620e2ce7df33 service neutron] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:04.303802 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:04.304416 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] REQUEST_METHOD: `GET` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:04.305221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:04.305221 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:04.306775 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:04.318876 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:04.320196 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:04.330540 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1459/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:56:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 14:56:04.456620 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:04.458048 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:04.473203 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaabb-b1e1-4100-87b0-33b404b3be62 service placement] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:04.478509 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:04.479142 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:04.479420 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:04.479694 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] PATH_INFO: `/v3/users/24992429cbbd47f3958e0de1be93916a` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:04.480832 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24992429cbbd47f3958e0de1be93916a)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:04.482764 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:04.648483 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24992429cbbd47f3958e0de1be93916a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:04.660921 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24992429cbbd47f3958e0de1be93916a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:04.706780 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24992429cbbd47f3958e0de1be93916a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:04.714615 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24992429cbbd47f3958e0de1be93916a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:04.715257 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24992429cbbd47f3958e0de1be93916a'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:04.721297 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1446/2905] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:56:04 2022] DELETE /identity/v3/users/24992429cbbd47f3958e0de1be93916a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:56:04.948896 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:04.964048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e26ed-a3a3-4998-b033-3ad39982448a service placement] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:05.022191 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:05.022829 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:05.023108 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:05.023378 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] PATH_INFO: `/v3/projects/be1df25ae6404f33b88dda96469e9a7e` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:05.024522 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1df25ae6404f33b88dda96469e9a7e)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:05.026449 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:05.049715 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:SELECT=100 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:56:05.106901 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1df25ae6404f33b88dda96469e9a7e', 'request_id': 'req-ff4fc466-3186-4afe-9e26-ed5ca61fc806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.113144 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1df25ae6404f33b88dda96469e9a7e', 'request_id': 'req-ff4fc466-3186-4afe-9e26-ed5ca61fc806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.157877 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1df25ae6404f33b88dda96469e9a7e', 'request_id': 'req-ff4fc466-3186-4afe-9e26-ed5ca61fc806'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.169457 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] The token cache is being invalidate because project be1df25ae6404f33b88dda96469e9a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:56:05.169745 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.170383 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.171906 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1460/2906] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:56:04 2022] DELETE /identity/v3/projects/be1df25ae6404f33b88dda96469e9a7e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:56:05.175894 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:05.177528 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:05.190446 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b3ded-f07e-4313-bef9-7572516ec441 admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:05.211867 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:56:05.248140 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:05.248970 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:05.249339 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:05.249835 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] PATH_INFO: `/v3/users/03b0466e477c421aac4d70971cf01532` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:05.251403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b0466e477c421aac4d70971cf01532)` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:05.253364 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:05.457405 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b0466e477c421aac4d70971cf01532'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.469403 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b0466e477c421aac4d70971cf01532'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.558245 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b0466e477c421aac4d70971cf01532'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.565641 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b0466e477c421aac4d70971cf01532'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.566302 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.notifications [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b0466e477c421aac4d70971cf01532'} {{(pid=80331) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:05.573067 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1447/2907] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Jul 29 14:56:05 2022] DELETE /identity/v3/users/03b0466e477c421aac4d70971cf01532 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:56:05.828481 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Authenticating user token {{(pid=80330) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:56:05.832526 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.fernet_utils [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80330) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:56:05.843048 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fc466-3186-4afe-9e26-ed5ca61fc806 admin admin] Validating token access rules against request {{(pid=80330) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:56:05.901253 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c8c97aa5a724434b6ecf6c70f044b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70f7d9422ed44219ac78a91471e7dbb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80330) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:56:05.906818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:56:05.906818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:56:05.906818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] PATH_INFO: `/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1` {{(pid=80330) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:56:05.906818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f78b0bf8e2d4154aa1bd59e91aa95f1)` {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:56:05.906818 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] RBAC: Authorization granted {{(pid=80330) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:56:06.009788 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f78b0bf8e2d4154aa1bd59e91aa95f1', 'request_id': 'req-a11c59ce-f5ab-4808-a9a9-750976da6069'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:06.016429 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f78b0bf8e2d4154aa1bd59e91aa95f1', 'request_id': 'req-a11c59ce-f5ab-4808-a9a9-750976da6069'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:06.104451 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f78b0bf8e2d4154aa1bd59e91aa95f1', 'request_id': 'req-a11c59ce-f5ab-4808-a9a9-750976da6069'} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:06.115741 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] The token cache is being invalidate because project 9f78b0bf8e2d4154aa1bd59e91aa95f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80330) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 29 14:56:06.116032 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:06.116678 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG keystone.notifications [None req-a11c59ce-f5ab-4808-a9a9-750976da6069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80330) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 29 14:56:06.118255 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: [pid: 80330|app: 0|req: 1461/2908] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Jul 29 14:56:05 2022] DELETE /identity/v3/projects/9f78b0bf8e2d4154aa1bd59e91aa95f1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 29 14:56:15.587312 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:56:16.111789 dv-cinder-799871-49-iscsi devstack@keystone.service[80330]: DEBUG dbcounter [-] [80330] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80330) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 29 14:59:59.211204 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Authenticating user token {{(pid=80331) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 29 14:59:59.212635 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:59:59.284721 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78880b66-13e4-4efb-a116-88b130d2383e admin admin] Validating token access rules against request {{(pid=80331) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 29 14:59:59.345380 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60aee4f7a30845f69db2e71f02fa139b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77d5d61ad1ff4826b55380d74a48285a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80331) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 29 14:59:59.345999 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] REQUEST_METHOD: `GET` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 29 14:59:59.346283 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] SCRIPT_NAME: `/identity` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 29 14:59:59.346581 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80331) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 29 14:59:59.348367 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.fernet_utils [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80331) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 29 14:59:59.360093 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 29 14:59:59.361702 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7183c-b649-469d-aeb3-7c8a9d76df8a service placement] RBAC: Authorization granted {{(pid=80331) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 29 14:59:59.371443 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: [pid: 80331|app: 0|req: 1448/2909] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Jul 29 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 29 15:00:09.367234 dv-cinder-799871-49-iscsi devstack@keystone.service[80331]: DEBUG dbcounter [-] [80331] Writing DB stats keystone:SELECT=58 {{(pid=80331) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}